paper-with-me

홈 › Papers

Automate Knowledge Concept Tagging on Math Questions with LLMs

2024-03-26 · Hang Li, Tianlong Xu, Jiliang Tang, Qingsong Wen

Knowledge concept tagging for questions plays a crucial role in contemporary intelligent educational applications, including learning progress diagnosis, practice question recommendations, and course content organization. Traditionally, these annotations have been conducted manually with help from pedagogical experts, as the task requires not only a strong semantic understanding of both question stems and knowledge definitions but also deep insights into connecting question-solving logic with corresponding knowledge concepts. In this paper, we explore automating the tagging task using Large Language Models (LLMs), in response to the inability of prior manual methods to meet the rapidly growing demand for concept tagging in questions posed by advanced educational applications. Moreover, the zero/few-shot learning capability of LLMs makes them well-suited for application in educational scenarios, which often face challenges in collecting large-scale, expertise-annotated datasets. By conducting extensive experiments with a variety of representative LLMs, we demonstrate that LLMs are a promising tool for concept tagging in math questions. Furthermore, through case studies examining the results from different LLMs, we draw some empirical conclusions about the key factors for success in applying LLMs to the automatic concept tagging task.

📄 PDF Abstract BibTeX arXiv:2403.17281

Code (0)

등록된 구현이 없습니다.

Tasks

Few-Shot LearningMath

Similar Papers 제목 키워드 기반

Knowledge Tagging System on Math Questions via LLMs with Flexible Demonstration Retriever

2024-06-19 · Hang Li, Tianlong Xu, Jiliang Tang, Qingsong Wen

Knowledge tagging for questions plays a crucial role in contemporary intelligent educational applications, including learning progress diagnosis, practice question recommendations, and course content organization. Tradit…

MathSemantic SimilaritySemantic Textual Similarity

Knowledge Tagging with Large Language Model based Multi-Agent System

2024-09-12 · Hang Li, Tianlong Xu, Ethan Chang, Qingsong Wen

Knowledge tagging for questions is vital in modern intelligent educational applications, including learning progress diagnosis, practice question recommendations, and course content organization. Traditionally, these ann…

Language ModelingLanguage ModellingLarge Language ModelMath

Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing

2024-10-02 · Yilmazcan Ozyurt, Stefan Feuerriegel, Mrinmaya Sachan

Knowledge tracing (KT) is a popular approach for modeling students' learning progress over time, which can enable more personalized and adaptive learning. However, existing KT approaches face two major limitations: (1) t…

Contrastive LearningKnowledge TracingMathRepresentation Learning

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions

2023-10-03 · Naiming Liu, Shashank Sonkar, Zichao Wang, Simon Woodhead 외

We propose novel evaluations for mathematical reasoning capabilities of Large Language Models (LLMs) based on mathematical misconceptions. Our primary approach is to simulate LLMs as a novice learner and an expert tutor,…

MathMathematical ReasoningMisconceptions

DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions

2024-06-27 · Nigel Fernandez, Alexander Scarlatos, Wanyong Feng, Simon Woodhead 외

High-quality distractors are crucial to both the assessment and pedagogical value of multiple-choice questions (MCQs), where manually crafting ones that anticipate knowledge deficiencies or misconceptions among real stud…

Distractor GenerationMathMisconceptionsMultiple-choice