Investigating Text Simplification Evaluation
Modern text simplification (TS) heavily relies on the availability of gold standard data to build machine learning models. However, existing studies show that parallel TS corpora contain inaccurate simplifications and incorrect alignments. Additionally, evaluation is usually performed by using metrics such as BLEU or SARI to compare system output to the gold standard. A major limitation is that these metrics do not match human judgements and the performance on different datasets and linguistic phenomena vary greatly. Furthermore, our research shows that the test and training subsets of parallel datasets differ significantly. In this work, we investigate existing TS corpora, providing new insights that will motivate the improvement of existing state-of-the-art TS evaluation methods. Our contributions include the analysis of TS corpora based on existing modifications used for simplification and an empirical study on TS models performance by using better-distributed datasets. We demonstrate that by improving the distribution of TS datasets, we can build more robust TS models.
Code (1)
Tasks
Text SimplificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Text Simplification for Comprehension-based Question-Answering
Text simplification is the process of splitting and rephrasing a sentence to a sequence of sentences making it easier to read and understand while preserving the content and approximating the original meaning. Text simpl…
Machine TranslationQuestion AnsweringSemantic Role LabelingSentence+2Automatic Simplification of Common Vulnerabilities and Exposures Descriptions
Understanding cyber security is increasingly important for individuals and organizations. However, a lot of information related to cyber security can be difficult to understand to those not familiar with the topic. In th…
Text SimplificationInvestigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts
Biomedical literature often uses complex language and inaccessible professional terminologies. That is why simplification plays an important role in improving public health literacy. Applying Natural Language Processing …
DecoderText SimplificationLENS: A Learnable Evaluation Metric for Text Simplification
Training learnable metrics using modern language models has recently emerged as a promising method for the automatic evaluation of machine translation. However, existing human evaluation datasets for text simplification …
Machine TranslationText SimplificationSEMA: Text Simplification Evaluation through Semantic Alignment
Text simplification is an important branch of natural language processing. At present, methods used to evaluate the semantic retention of text simplification are mostly based on string matching. We propose the SEMA (text…
Text Simplification