paper-with-me

홈 › Papers

A Spectral Approach to Item Response Theory

2022-10-09 · Duc Nguyen, Anderson Zhang

The Rasch model is one of the most fundamental models in \emph{item response theory} and has wide-ranging applications from education testing to recommendation systems. In a universe with $n$ users and $m$ items, the Rasch model assumes that the binary response $X_{li} \in \{0,1\}$ of a user $l$ with parameter $\theta^*_l$ to an item $i$ with parameter $\beta^*_i$ (e.g., a user likes a movie, a student correctly solves a problem) is distributed as $\Pr(X_{li}=1) = 1/(1 + \exp{-(\theta^*_l - \beta^*_i)})$. In this paper, we propose a \emph{new item estimation} algorithm for this celebrated model (i.e., to estimate $\beta^*$). The core of our algorithm is the computation of the stationary distribution of a Markov chain defined on an item-item graph. We complement our algorithmic contributions with finite-sample error guarantees, the first of their kind in the literature, showing that our algorithm is consistent and enjoys favorable optimality properties. We discuss practical modifications to accelerate and robustify the algorithm that practitioners can adopt. Experiments on synthetic and real-life datasets, ranging from small education testing datasets to large recommendation systems datasets show that our algorithm is scalable, accurate, and competitive with the most commonly used methods in the literature.

📄 PDF Abstract BibTeX arXiv:2210.04317

Code (0)

등록된 구현이 없습니다.

Tasks

Recommendation Systems

Similar Papers 제목 키워드 기반

Optimal and Private Learning from Human Response Data

2023-03-10 · Duc Nguyen, Anderson Y. Zhang

Item response theory (IRT) is the study of how people make probabilistic decisions, with diverse applications in education testing, recommendation systems, among others. The Rasch model of binary response data, one of th…

Privacy PreservingRecommendation Systems

Matrix Decomposition Perspective for Accuracy Assessment of Item Response Theory

2022-03-07 · Hideo Hirose

The item response theory obtains the estimates and their confidence intervals for parameters of abilities of examinees and difficulties of problems by using the observed item response matrix consisting of 0/1 value eleme…

SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction

2025-07-07 · Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge 외

Item (question) difficulties play a crucial role in educational assessments, enabling accurate and efficient assessment of student abilities and personalization to maximize learning outcomes. Traditionally, estimating it…

py-irt: A Scalable Item Response Theory Library for Python

2022-03-02 · John P. Lalor, Pedro Rodriguez

py-irt is a Python library for fitting Bayesian Item Response Theory (IRT) models. py-irt estimates latent traits of subjects and items, making it appropriate for use in IRT tasks as well as ideal-point models. py-irt is…

GPU

Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theory

2025-05-21 · Hongli Zhou, Hui Huang, Ziqing Zhao, Lvyuan Han 외

The evaluation of large language models (LLMs) via benchmarks is widespread, yet inconsistencies between different leaderboards and poor separability among top models raise concerns about their ability to accurately refl…

BenchmarkingLanguage ModelingLanguage ModellingLarge Language Model