Controllable Sentence Simplification with a Unified Text-to-Text Transfer Transformer
Recently, a large pre-trained language model called T5 (A Unified Text-to-Text Transfer Transformer) has achieved state-of-the-art performance in many NLP tasks. However, no study has been found using this pre-trained model on Text Simplification. Therefore in this paper, we explore the use of T5 fine-tuning on Text Simplification combining with a controllable mechanism to regulate the system outputs that can help generate adapted text for different target audiences. Our experiments show that our model achieves remarkable results with gains of between +0.69 and +1.41 over the current state-of-the-art (BART+ACCESS). We argue that using a pre-trained model such as T5, trained on several tasks with large amounts of data, can help improve Text Simplification.
Code (1)
Tasks
Language ModelingLanguage ModellingSentenceText SimplificationSimilar Papers 제목 키워드 기반
Combining Paraphrase Pre-trained Model and Controllable Rules for Unsupervised Sentence Simplification
Although neural sequence-to-sequence models for sentence simplification achieve some progress, they still suffer from the data sparsity problem and are lack of controllability. This paper proposes a two-stage approach fo…
SentenceText SimplificationText SummarizationContext-Aware Document Simplification
To date, most work on text simplification has focused on sentence-level inputs. Early attempts at document simplification merely applied these approaches iteratively over the sentences of a document. However, this fails …
SentenceText SimplificationControllable Text Simplification with Lexical Constraint Loss
We propose a method to control the level of a sentence in a text simplification task. Text simplification is a monolingual translation task translating a complex sentence into a simpler and easier to understand the alter…
SentenceText SimplificationTranslationControllable Sentence Simplification via Operation Classification
Different types of transformations have been used to model sentence simplification ranging from mainly local operations such as phrasal or lexical rewriting, deletion and re-ordering to the more global affecting the whol…
ClassificationSentenceControllable Sentence Simplification
Text simplification aims at making a text easier to read and understand by simplifying grammar and structure while keeping the underlying information identical. It is often considered an all-purpose generic task where th…
SentenceText Simplification