Multi-Faceted Question Complexity Estimation Targeting Topic Domain-Specificity
Question difficulty estimation remains a multifaceted challenge in educational and assessment settings. Traditional approaches often focus on surface-level linguistic features or learner comprehension levels, neglecting the intricate interplay of factors contributing to question complexity. This paper presents a novel framework for domain-specific question difficulty estimation, leveraging a suite of NLP techniques and knowledge graph analysis. We introduce four key parameters: Topic Retrieval Cost, Topic Salience, Topic Coherence, and Topic Superficiality, each capturing a distinct facet of question complexity within a given subject domain. These parameters are operationalized through topic modelling, knowledge graph analysis, and information retrieval techniques. A model trained on these features demonstrates the efficacy of our approach in predicting question difficulty. By operationalizing these parameters, our framework offers a novel approach to question complexity estimation, paving the way for more effective question generation, assessment design, and adaptive learning systems across diverse academic disciplines.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalQuestion GenerationQuestion-GenerationRetrievalSpecificityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Critical Evaluation of Evaluations for Long-form Question Answering
Long-form question answering (LFQA) enables answering a wide range of questions, but its flexibility poses enormous challenges for evaluation. We perform the first targeted study of the evaluation of long-form answers, c…
FormLong Form Question AnsweringQuestion AnsweringText GenerationMultifacetEval: Multifaceted Evaluation to Probe LLMs in Mastering Medical Knowledge
Large language models (LLMs) have excelled across domains, also delivering notable performance on the medical evaluation benchmarks, such as MedQA. However, there still exists a significant gap between the reported perfo…
MedQAMultifaceted Improvements for Conversational Open-Domain Question Answering
Open-domain question answering (OpenQA) is an important branch of textual QA which discovers answers for the given questions based on a large number of unstructured documents. Effectively mining correct answers from the …
Open-Domain Question AnsweringQuestion AnsweringRetrievalQuery Refinement Prompts for Closed-Book Long-Form Question Answering
Large language models (LLMs) have been shown to perform well in answering questions and in producing long-form texts, both in few-shot closed-book settings. While the former can be validated using well-known evaluation m…
FormLong Form Question AnsweringQuestion AnsweringMeteor: Mamba-based Traversal of Rationale for Large Language and Vision Models
The rapid development of large language and vision models (LLVMs) has been driven by advances in visual instruction tuning. Recently, open-source LLVMs have curated high-quality visual instruction tuning datasets and uti…
Common Sense ReasoningLanguage ModellingMambaMath+2