AGenT Zero: Zero-shot Automatic Multiple-Choice Question Generation for Skill Assessments
Multiple-choice questions (MCQs) offer the most promising avenue for skill evaluation in the era of virtual education and job recruiting, where traditional performance-based alternatives such as projects and essays have become less viable, and grading resources are constrained. The automated generation of MCQs would allow assessment creation at scale. Recent advances in natural language processing have given rise to many complex question generation methods. However, the few methods that produce deployable results in specific domains require a large amount of domain-specific training data that can be very costly to acquire. Our work provides an initial foray into MCQ generation under high data-acquisition cost scenarios by strategically emphasizing paraphrasing the question context (compared to the task). In addition to maintaining semantic similarity between the question-answer pairs, our pipeline, which we call AGenT Zero, consists of only pre-trained models and requires no fine-tuning, minimizing data acquisition costs for question generation. AGenT Zero successfully outperforms other pre-trained methods in fluency and semantic similarity. Additionally, with some small changes, our assessment pipeline can be generalized to a broader question and answer space, including short answer or fill in the blank questions.
Code (0)
등록된 구현이 없습니다.
Tasks
Multiple-choiceQuestion GenerationQuestion-GenerationSemantic SimilaritySemantic Textual SimilaritySimilar Papers 제목 키워드 기반
Automatic Curriculum Design for Zero-Shot Human-AI Coordination
Zero-shot human-AI coordination is the training of an ego-agent to coordinate with humans without using human data. Most studies on zero-shot human-AI coordination have focused on enhancing the ego-agent's coordination a…
A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition
Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. This task presents substantial challenges due to minimal human intervention. Recent work has adapted larg…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+3Zero and Few-Shot Localization of Task-Oriented Dialogue Agents with a Distilled Representation
Task-oriented Dialogue (ToD) agents are mostly limited to a few widely-spoken languages, mainly due to the high cost of acquiring training data for each language. Existing low-cost approaches that rely on cross-lingual e…
Dialogue State TrackingMachine TranslationTranslationA Unified approach for Conventional Zero-shot, Generalized Zero-shot and Few-shot Learning
Prevalent techniques in zero-shot learning do not generalize well to other related problem scenarios. Here, we present a unified approach for conventional zero-shot, generalized zero-shot and few-shot learning problems. …
Few-Shot LearningGeneralized Zero-Shot LearningOne-Shot LearningZero-Shot LearningImproving Zero-Shot Offline RL via Behavioral Task Sampling
Offline zero-shot reinforcement learning (RL) aims to learn agents that optimize unseen reward functions without additional environment interaction. The standard approach to this problem trains task-conditioned policies …
Zero-shot GeneralizationReinforcement LearningOffline RL