paper-with-me

홈 › Papers

Revisiting Alternative Experimental Settings for Evaluating Top-N Item Recommendation Algorithms

2020-10-09 · Wayne Xin Zhao, Junhua Chen, Pengfei Wang, Qi Gu, Ji-Rong Wen

Top-N item recommendation has been a widely studied task from implicit feedback. Although much progress has been made with neural methods, there is increasing concern on appropriate evaluation of recommendation algorithms. In this paper, we revisit alternative experimental settings for evaluating top-N recommendation algorithms, considering three important factors, namely dataset splitting, sampled metrics and domain selection. We select eight representative recommendation algorithms (covering both traditional and neural methods) and construct extensive experiments on a very large dataset. By carefully revisiting different options, we make several important findings on the three factors, which directly provide useful suggestions on how to appropriately set up the experiments for top-N item recommendation.

📄 PDF Abstract BibTeX arXiv:2010.04484

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

What Else Would I Like? A User Simulator using Alternatives for Improved Evaluation of Fashion Conversational Recommendation Systems

2024-01-11 · Maria Vlachou, Craig Macdonald

In Conversational Recommendation Systems (CRS), a user can provide feedback on recommended items at each interaction turn, leading the CRS towards more desirable recommendations. Currently, different types of CRS offer v…

Conversational RecommendationImage CaptioningRecommendation Systems

Revisiting Gradient Staleness: Evaluating Distance Metrics for Asynchronous Federated Learning Aggregation

2026-03-09 · Patrick Wilhelm, Odej Kao arxiv

In asynchronous federated learning (FL), client devices send updates to a central server at varying times based on their computational speed, often using stale versions of the global model. This staleness can degrade the…

Federated Learning

ConvFormer: Revisiting Transformer for Sequential User Modeling

2023-08-05 · Hao Wang, Jianxun Lian, Mingqi Wu, Haoxuan Li 외

Sequential user modeling, a critical task in personalized recommender systems, focuses on predicting the next item a user would prefer, requiring a deep understanding of user behavior sequences. Despite the remarkable su…

Recommendation Systems

Estimation of Fair Ranking Metrics with Incomplete Judgments

2021-08-11 · Ömer Kırnap, Fernando Diaz, Asia Biega, Michael Ekstrand 외

There is increasing attention to evaluating the fairness of search system ranking decisions. These metrics often consider the membership of items to particular groups, often identified using protected attributes such as …

AttributeFairness

Fashion-AlterEval: A Dataset for Improved Evaluation of Conversational Recommendation Systems with Alternative Relevant Items

2025-07-24 · Maria Vlachou arxiv

In Conversational Recommendation Systems (CRS), a user provides feedback on recommended items at each turn, leading the CRS towards improved recommendations. Due to the need for a large amount of data, a user simulator i…

Recommendation Systems