Question Difficulty -- How to Estimate Without Norming, How to Use for Automated Grading
Question difficulty estimates guide test creation, but are too costly for small-scale testing. We empirically verify that Bloom{'}s Taxonomy, a standard tool for difficulty estimation during question creation, reliably predicts question difficulty observed after testing in a short-answer corpus. We also find that difficulty is mirrored in the amount of variation in student answers, which can be computed before grading. We show that question difficulty and its approximations are useful for \textit{automated grading}, allowing us to identify the optimal feature set for grading each question even in an unseen-question setting.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Can LLMs capture stable human-generated sentence entropy measures?
Predicting upcoming words is a core mechanism of language comprehension and may be quantified using Shannon entropy. There is currently no empirical consensus on how many human responses are required to obtain stable and…
Exploring the Potential of Large Language Models for Estimating the Reading Comprehension Question Difficulty
Reading comprehension is a key for individual success, yet the assessment of question difficulty remains challenging due to the extensive human annotation and large-scale testing required by traditional methods such as l…
Reading ComprehensionA-SEA3L-QA: A Fully Automated Self-Evolving, Adversarial Workflow for Arabic Long-Context Question-Answer Generation
We present an end-to-end, self-evolving adversarial workflow for long-context Question-Answer (QA) Generation in Arabic. By orchestrating multiple specialized LVLMs: a question generator, an evaluator, and a swarm of ans…
Question GenerationAnswer GenerationCan LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Prediction
Accurate estimation of item (question or task) difficulty is critical for educational assessment but suffers from the cold start problem. While Large Language Models demonstrate superhuman problem-solving capabilities, i…
Mathematical ReasoningThe LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations
Estimating the difficulty of input questions as perceived by large language models (LLMs) is essential for accurate performance evaluation and adaptive inference. Existing methods typically rely on repeated response samp…