Evaluation methodologies in Automatic Question Generation 2013-2018
In the last few years Automatic Question Generation (AQG) has attracted increasing interest. In this paper we survey the evaluation methodologies used in AQG. Based on a sample of 37 papers, our research shows that the systems{'} development has not been accompanied by similar developments in the methodologies used for the systems{'} evaluation. Indeed, in the papers we examine here, we find a wide variety of both intrinsic and extrinsic evaluation methodologies. Such diverse evaluation practices make it difficult to reliably compare the quality of different generation systems. Our study suggests that, given the rapidly increasing level of research in the area, a common framework is urgently needed to compare the performance of AQG systems and NLG systems more generally.
Code (0)
등록된 구현이 없습니다.
Tasks
Question GenerationQuestion-GenerationText GenerationSimilar Papers 제목 키워드 기반
Story Generation from Visual Inputs: Techniques, Related Tasks, and Challenges
Creating engaging narratives from visual data is crucial for automated digital media consumption, assistive technologies, and interactive entertainment. This survey covers methodologies used in the generation of these na…
Question AnsweringStory GenerationSurveyVideo Captioning+1Recent Advances in Neural Question Generation
Emerging research in Neural Question Generation (NQG) has started to integrate a larger variety of inputs, and generating questions requiring higher levels of cognition. These trends point to NQG as a bellwether for NLP,…
Question GenerationQuestion-GenerationA RAG Approach for Generating Competency Questions in Ontology Engineering
Competency question (CQ) formulation is central to several ontology development and evaluation methodologies. Traditionally, the task of crafting these competency questions heavily relies on the effort of domain experts …
Knowledge GraphsRAGRetrieval-augmented GenerationAutomatic Opinion Question Generation
We study the problem of opinion question generation from sentences with the help of community-based question answering systems. For this purpose, we use a sequence to sequence attentional model, and we adopt coverage mec…
Community Question AnsweringQuestion AnsweringQuestion GenerationQuestion-Generation+2A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations
Recent advances in deep learning have significantly enhanced generative AI capabilities across text, images, and audio. However, automatically evaluating the quality of these generated outputs presents ongoing challenges…
Audio GenerationText Generation