홈 › Papers

Improving Unsupervised Sentence Simplification Using Fine-Tuned Masked Language Models

Simple word suggestion in unsupervised sentence simplification (SS) methods is mostly done independently of the context. The idea of adapting and fine-tuning a context-aware model on simple data for improving performance has been discussed but not practiced. In this paper, we propose a framework involving fine-tuning a pre-trained BERT masked language model on simple English corpora to aid SS. Our analysis of public test data shows that fine-tuning on any set of simple sentences do not necessarily yield better simplifications but generally makes improvements. To tackle this issue, we propose a self-supervised framework which is composed of a labeling method that conducts an estimate about the \textit{usefulness} of each training sample, paired with a simple linear classifier that decides the inclusion of a given sentence in the fine-tuning process. The fine-tuned BERT will be used in an iterative edit-based unsupervised SS model to provide contextual word suggestions. The results show that our data selection approach can improve simplifications as much as having a simple-to-complex parallel corpus.

Code (0)

등록된 구현이 없습니다.

Tasks

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반