Evaluation of Automatically Generated Pronoun Reference Questions
This study provides a detailed analysis of evaluation of English pronoun reference questions which are created automatically by machine. Pronoun reference questions are multiple choice questions that ask test takers to choose an antecedent of a target pronoun in a reading passage from four options. The evaluation was performed from two perspectives: the perspective of English teachers and that of English learners. Item analysis suggests that machine-generated questions achieve comparable quality with human-made questions. Correlation analysis revealed a strong correlation between the scores of machine-generated questions and that of human-made questions.
Code (0)
등록된 구현이 없습니다.
Tasks
Multiple-choiceReading ComprehensionSimilar Papers 제목 키워드 기반
Automatic Question Generation using Relative Pronouns and Adverbs
This paper presents a system that automatically generates multiple, natural language questions using relative pronouns and relative adverbs from complex English sentences. Our system is syntax-based, runs on dependency p…
DescriptiveDialogue GenerationInformation RetrievalQuestion Answering+4Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
Practical applications of abstractive summarization models are limited by frequent factual inconsistencies with respect to their input. Existing automatic evaluation metrics for summarization are largely insensitive to s…
Abstractive Text SummarizationRapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese
The creation of instruction data and evaluation benchmarks for serving Large language models often involves enormous human annotation. This issue becomes particularly pronounced when rapidly developing such resources for…
Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems
A key distinguishing feature of conversational recommender systems over traditional recommender systems is their ability to elicit user preferences using natural language. Currently, the predominant approach to preferenc…
Machine TranslationQuestion GenerationQuestion-GenerationRecommendation SystemsCan LLMs Detect Ambiguous Plural Reference? An Analysis of Split-Antecedent and Mereological Reference
Our goal is to study how LLMs represent and interpret plural reference in ambiguous and unambiguous contexts. We ask the following research questions: (1) Do LLMs exhibit human-like preferences in representing plural ref…