Semantic Structural Evaluation for Text Simplification
Current measures for evaluating text simplification systems focus on evaluating lexical text aspects, neglecting its structural aspects. In this paper we propose the first measure to address structural aspects of text simplification, called SAMSA. It leverages recent advances in semantic parsing to assess simplification quality by decomposing the input based on its semantic structure and comparing it to the output. SAMSA provides a reference-less automatic evaluation procedure, avoiding the problems that reference-based methods face due to the vast space of valid simplifications for a given sentence. Our human evaluation experiments show both SAMSA's substantial correlation with human judgments, as well as the deficiency of existing reference-based measures in evaluating structural simplification.
Code (1)
Tasks
Semantic ParsingSentenceText SimplificationvalidSimilar Papers 제목 키워드 기반
Simple and Effective Text Simplification Using Semantic and Neural Methods
Sentence splitting is a major simplification operator. Here we present a simple and efficient splitting algorithm based on an automatic semantic parser. After splitting, the text is amenable for further fine-tuned simpli…
Machine TranslationSemantic ParsingSentenceText Simplification+1SEMA: Text Simplification Evaluation through Semantic Alignment
Text simplification is an important branch of natural language processing. At present, methods used to evaluate the semantic retention of text simplification are mostly based on string matching. We propose the SEMA (text…
Text SimplificationBLEU is Not Suitable for the Evaluation of Text Simplification
BLEU is widely considered to be an informative metric for text-to-text generation, including Text Simplification (TS). TS includes both lexical and structural aspects. In this paper we show that BLEU is not suitable for …
SentenceText GenerationText SimplificationSimplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
Large language models (LLMs) enable strong text generation, and in general there is a practical tradeoff between fine-tuning and prompt engineering. We introduce Simplify-This, a comparative study evaluating both paradig…
Text SimplificationSemantic SimilarityPrompt EngineeringText GenerationTransforming Complex Sentences into a Semantic Hierarchy
We present an approach for recursively splitting and rephrasing complex English sentences into a novel semantic hierarchy of simplified sentences, with each of them presenting a more regular structure that may facilitate…
Machine TranslationText SimplificationTranslation