Multiple-Choice Question Generation Using Large Language Models: Methodology and Educator Insights
Integrating Artificial Intelligence (AI) in educational settings has brought new learning approaches, transforming the practices of both students and educators. Among the various technologies driving this transformation, Large Language Models (LLMs) have emerged as powerful tools for creating educational materials and question answering, but there are still space for new applications. Educators commonly use Multiple-Choice Questions (MCQs) to assess student knowledge, but manually generating these questions is resource-intensive and requires significant time and cognitive effort. In our opinion, LLMs offer a promising solution to these challenges. This paper presents a novel comparative analysis of three widely known LLMs - Llama 2, Mistral, and GPT-3.5 - to explore their potential for creating informative and challenging MCQs. In our approach, we do not rely on the knowledge of the LLM, but we inject the knowledge into the prompt to contrast the hallucinations, giving the educators control over the test's source text, too. Our experiment involving 21 educators shows that GPT-3.5 generates the most effective MCQs across several known metrics. Additionally, it shows that there is still some reluctance to adopt AI in the educational field. This study sheds light on the potential of LLMs to generate MCQs and improve the educational experience, providing valuable insights for the future.
Code (0)
등록된 구현이 없습니다.
Tasks
Multiple-choiceQuestion AnsweringQuestion GenerationQuestion-GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization
Difficulty-controllable question generation for reading comprehension has gained significant attention in the field of education as a fundamental tool for adaptive learning support. Although several neural question gener…
Reading ComprehensionQuestion GenerationBetter Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering
For the field of education, being able to generate semantically correct and educationally relevant multiple choice questions (MCQs) could have a large impact. While question generation itself is an active research topic,…
Distractor GenerationLanguage ModelingLanguage ModellingMultiple-choice+4The Benchmark Illusion: Pruned LLMs Can Pass Multiple Choice but Fail to Answer
Compressing large language models reduces memory use and inference cost, but it can also create failures that standard benchmarks miss. A pruned model may still perform well on multiple-choice evaluations, yet fail to an…
Question AnsweringAutomated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation
The rapid development of vision language models (VLMs) demands rigorous and reliable evaluation. However, current visual question answering (VQA) benchmarks often depend on open-ended questions, making accurate evaluatio…
Language Model EvaluationLanguage ModelingLanguage ModellingMultiple-choice+3Generating Questions and Multiple-Choice Answers using Semantic Analysis of Texts
We present a novel approach to automated question generation that improves upon prior work both from a technology perspective and from an assessment perspective. Our system is aimed at engaging language learners by gener…
coreference-resolutionCoreference ResolutionMultiple-choiceQuestion Generation+3