Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback Datasets
The recommender systems algorithm selection problem for ranking prediction on implicit feedback datasets is under-explored. Traditional approaches in recommender systems algorithm selection focus predominantly on rating prediction on explicit feedback datasets, leaving a research gap for ranking prediction on implicit feedback datasets. Algorithm selection is a critical challenge for nearly every practitioner in recommender systems. In this work, we take the first steps toward addressing this research gap. We evaluate the NDCG@10 of 24 recommender systems algorithms, each with two hyperparameter configurations, on 72 recommender systems datasets. We train four optimized machine-learning meta-models and one automated machine-learning meta-model with three different settings on the resulting meta-dataset. Our results show that the predictions of all tested meta-models exhibit a median Spearman correlation ranging from 0.857 to 0.918 with the ground truth. We show that the median Spearman correlation between meta-model predictions and the ground truth increases by an average of 0.124 when the meta-model is optimized to predict the ranking of algorithms instead of their performance. Furthermore, in terms of predicting the best algorithm for an unknown dataset, we demonstrate that the best optimized traditional meta-model, e.g., XGBoost, achieves a recall of 48.6%, outperforming the best tested automated machine learning meta-model, e.g., AutoGluon, which achieves a recall of 47.2%.
Code (0)
등록된 구현이 없습니다.
Tasks
Recommendation SystemsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revealing the Hidden Impact of Top-N Metrics on Optimization in Recommender Systems
The hyperparameters of recommender systems for top-n predictions are typically optimized to enhance the predictive performance of algorithms. Thereby, the optimization algorithm, e.g., grid search or random search, searc…
Recommendation SystemsRe-RankingSkellam Rank: Fair Learning to Rank Algorithm Based on Poisson Process and Skellam Distribution for Recommender Systems
Recommender system is a widely adopted technology in a diversified class of product lines. Modern day recommender system approaches include matrix factorization, learning to rank and deep learning paradigms, etc. Unlike …
FairnessLearning-To-RankRecommendation SystemsSelection biasReinforcement Re-ranking with 2D Grid-based Recommendation Panels
Modern recommender systems usually present items as a streaming, one-dimensional ranking list. Recently there is a trend in e-commerce that the recommended items are organized grid-based panels with two dimensions where …
Recommendation SystemsRe-RankingExploring Data Splitting Strategies for the Evaluation of Recommendation Models
Effective methodologies for evaluating recommender systems are critical, so that such systems can be compared in a sound manner. A commonly overlooked aspect of recommender system evaluation is the selection of the data …
Recommendation SystemsRecRec: Algorithmic Recourse for Recommender Systems
Recommender systems play an essential role in the choices people make in domains such as entertainment, shopping, food, news, employment, and education. The machine learning models underlying these recommender systems ar…
Recommendation Systemsvalid