A Re-visit of the Popularity Baseline in Recommender Systems
Popularity is often included in experimental evaluation to provide a reference performance for a recommendation task. To understand how popularity baseline is defined and evaluated, we sample 12 papers from top-tier conferences including KDD, WWW, SIGIR, and RecSys, and 6 open source toolkits. We note that the widely adopted MostPop baseline simply ranks items based on the number of interactions in the training data. We argue that the current evaluation of popularity (i) does not reflect the popular items at the time when a user interacts with the system, and (ii) may recommend items released after a user's last interaction with the system. On the widely used MovieLens dataset, we show that the performance of popularity could be significantly improved by 70% or more, if we consider the popular items at the time point when a user interacts with the system. We further show that, on MovieLens dataset, the users having lower tendencies on movies tend to follow the crowd and rate more popular movies. Movie lovers who rate a large number of movies, rate movies based on their own preferences and interests. Through this study, we call for a re-visit of the popularity baseline in recommender system to better reflect its effectiveness.
Code (1)
Tasks
Recommendation SystemsSimilar Papers 제목 키워드 기반
Large Language Models as Recommender Systems: A Study of Popularity Bias
The issue of popularity bias -- where popular items are disproportionately recommended, overshadowing less popular but potentially relevant items -- remains a significant challenge in recommender systems. Recent advancem…
Movie RecommendationRecommendation SystemsThe Diversity Paradox revisited: Systemic Effects of Feedback Loops in Recommender Systems
Recommender systems shape individual choices through feedback loops in which user behavior and algorithmic recommendations coevolve over time. The systemic effects of these loops remain poorly understood, in part due to …
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Recommendation algorithms are susceptible to popularity bias: a tendency to recommend popular items even when they fail to meet user needs. A related issue is that the recommendation quality can vary by demographic group…
Enhancing Tourism Recommender Systems for Sustainable City Trips Using Retrieval-Augmented Generation
Tourism Recommender Systems (TRS) have traditionally focused on providing personalized travel suggestions, often prioritizing user preferences without considering broader sustainability goals. Integrating sustainability …
RAGRecommendation SystemsRerankingRetrieval+1Revisiting Self-Attentive Sequential Recommendation
Recommender systems are ubiquitous in on-line services to drive businesses. And many sequential recommender models were deployed in these systems to enhance personalization. The approach of using the transformer decoder …
DecoderRecommendation SystemsSequential Recommendation