Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training
Automatic text simplification systems help to reduce textual information barriers on the internet. However, for languages other than English, only few parallel data to train these systems exists. We propose a two-step approach to overcome this data scarcity issue. First, we fine-tuned language models on a corpus of German Easy Language, a specific style of German. Then, we used these models as decoders in a sequence-to-sequence simplification task. We show that the language models adapt to the style characteristics of Easy Language and output more accessible texts. Moreover, with the style-specific pre-training, we reduced the number of trainable parameters in text simplification models. Hence, less parallel data is sufficient for training. Our results indicate that pre-training on unaligned data can reduce the required parallel data while improving the performance on downstream tasks.
Code (1)
Tasks
Text SimplificationSimilar Papers 제목 키워드 기반
DEPLAIN: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification
Text simplification is an intralingual translation task in which documents, or sentences of a complex source text are simplified for a target audience. The success of automatic text simplification systems is highly depen…
2kSentenceText SimplificationBenchmarking Data-driven Automatic Text Simplification for German
Automatic text simplification is an active research area, and there are first systems for English, Spanish, Portuguese, and Italian. For German, no data-driven approach exists to this date, due to a lack of training data…
BenchmarkingMachine TranslationSentenceText Simplification+1Building a German/Simple German Parallel Corpus for Automatic Text Simplification
Data and Approaches for German Text simplification -- towards an Accessibility-enhanced Communication
This paper examines the current state-of-the-art of German text simplification, focusing on parallel and monolingual German corpora. It reviews neural language models for simplifying German texts and assesses their suita…
Text SimplificationPatient-friendly Clinical Notes: Towards a new Text Simplification Dataset
Automatic text simplification can help patients to better understand their own clinical notes. A major hurdle for the development of clinical text simplification methods is the lack of high quality resources. We report o…
Text Simplification