Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers
This paper presents the first multi-objective transformer model for constructing open cloze tests that exploits generation and discrimination capabilities to improve performance. Our model is further enhanced by tweaking its loss function and applying a post-processing re-ranking algorithm that improves overall test structure. Experiments using automatic and human evaluation show that our approach can achieve up to 82% accuracy according to experts, outperforming previous work and baselines. We also release a collection of high-quality open cloze tests along with sample system output and human annotations that can serve as a future benchmark.
Code (0)
등록된 구현이 없습니다.
Tasks
Overall - TestRe-RankingSimilar Papers 제목 키워드 기반
Entropy as a Proxy for Gap Complexity in Open Cloze Tests
This paper presents a pilot study of entropy as a measure of gap complexity in open cloze tests aimed at learners of English. Entropy is used to quantify the information content in each gap, which can be used to estimate…
CEPOC: The Cambridge Exams Publishing Open Cloze dataset
Open cloze tests are a standard type of exercise where examinees must complete a text by filling in the gaps without any given options to choose from. This paper presents the Cambridge Exams Publishing Open Cloze (CEPOC)…
text-classificationText ClassificationDualReward: A Dynamic Reinforcement Learning Framework for Cloze Tests Distractor Generation
This paper introduces DualReward, a novel reinforcement learning framework for automatic distractor generation in cloze tests. Unlike conventional approaches that rely primarily on supervised learning or static generativ…
Reinforcement LearningDistractor GenerationCloze TestCloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian
Story comprehension that involves complex causal and temporal relations is imperative in NLP, but previous studies have focused on English, leaving open the question of how the findings generalize to other languages, suc…
Cloze TestCross-Lingual TransferSentenceZero-Shot Cross-Lingual TransferCloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian
Story comprehension that involves complex causal and temporal relations is a critical task in NLP, but previous studies have focused predominantly on English, leaving open the question of how the findings generalize to o…
Cloze TestCross-Lingual TransferSentenceZero-Shot Cross-Lingual Transfer