Variational Item Response Theory: Fast, Accurate, and Expressive
Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger datasets pose a difficult speed / accuracy challenge to contemporary algorithms for fitting IRT models. We introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scaleable without sacrificing accuracy. Using this inference approach we then extend classic IRT with expressive Bayesian models of responses. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and improvements in imputing missing data. The algorithm implementation is open-source, and easily usable.
Code (1)
Tasks
Bayesian InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational Temporal IRT: Fast, Accurate, and Explainable Inference of Dynamic Learner Proficiency
Dynamic Item Response Models extend the standard Item Response Theory (IRT) to capture temporal dynamics in learner ability. While these models have the potential to allow instructional systems to actively monitor the ev…
Modeling Item Response Theory with Stochastic Variational Inference
Item Response Theory (IRT) is a ubiquitous model for understanding human behaviors and attitudes based on their responses to questions. Large modern datasets offer opportunities to capture more nuances in human behavior,…
Bayesian InferenceVariational InferenceA Deep Learning Algorithm for High-Dimensional Exploratory Item Factor Analysis
Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to…
Computational EfficiencyVariational InferenceVocal Bursts Intensity PredictionCovariance Matrices for Mean Field Variational Bayes
Mean Field Variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, it is well known that a major failing of MFVB is its (sometimes severe) underesti…
Lost in Benchmarks? Rethinking Large Language Model Benchmarking with Item Response Theory
The evaluation of large language models (LLMs) via benchmarks is widespread, yet inconsistencies between different leaderboards and poor separability among top models raise concerns about their ability to accurately refl…
BenchmarkingLanguage ModelingLanguage ModellingLarge Language Model