SplitLight: An Exploratory Toolkit for Recommender Systems Datasets and Splits
Offline evaluation of recommender systems is often affected by hidden, under-documented choices in data preparation. Seemingly minor decisions in filtering, handling repeats, cold-start treatment, and splitting strategy design can substantially reorder model rankings and undermine reproducibility and cross-paper comparability. In this paper, we introduce SplitLight, an open-source exploratory toolkit that enables researchers and practitioners designing preprocessing and splitting pipelines or reviewing external artifacts to make these decisions measurable, comparable, and reportable. Given an interaction log and derived split subsets, SplitLight analyzes core and temporal dataset statistics, characterizes repeat consumption patterns and timestamp anomalies, and diagnoses split validity, including temporal leakage, cold-user/item exposure, and distribution shifts. SplitLight further allows side-by-side comparison of alternative splitting strategies through comprehensive aggregated summaries and interactive visualizations. Delivered as both a Python toolkit and an interactive no-code interface, SplitLight produces audit summaries that justify evaluation protocols and support transparent, reliable, and comparable experimentation in recommender systems research and industry.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems
We present an extensible user simulation toolkit to facilitate automatic evaluation of conversational recommender systems. It builds on an established agenda-based approach and extends it with several novel elements, inc…
Recommendation SystemsText GenerationUser SimulationGraph4Rec: A Universal Toolkit with Graph Neural Networks for Recommender Systems
In recent years, owing to the outstanding performance in graph representation learning, graph neural network (GNN) techniques have gained considerable interests in many real-world scenarios, such as recommender systems a…
graph constructionGraph Neural NetworkGraph Representation LearningRecommendation Systems+1RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems
Robust machine learning is an increasingly important topic that focuses on developing models resilient to various forms of imperfect data. Due to the pervasiveness of recommender systems in online technologies, researche…
Recommendation SystemsWhere Do We Go From Here? Guidelines For Offline Recommender Evaluation
Various studies in recent years have pointed out large issues in the offline evaluation of recommender systems, making it difficult to assess whether true progress has been made. However, there has been little research i…
Hyperparameter OptimizationRecommendation SystemsUncertainty QuantificationRecAI: Leveraging Large Language Models for Next-Generation Recommender Systems
This paper introduces RecAI, a practical toolkit designed to augment or even revolutionize recommender systems with the advanced capabilities of Large Language Models (LLMs). RecAI provides a suite of tools, including Re…
AI AgentRecommendation Systems