Enhancing Pre-trained Language Model with Lexical Simplification
For both human readers and pre-trained language models (PrLMs), lexical diversity may lead to confusion and inaccuracy when understanding the underlying semantic meanings of given sentences. By substituting complex words with simple alternatives, lexical simplification (LS) is a recognized method to reduce such lexical diversity, and therefore to improve the understandability of sentences. In this paper, we leverage LS and propose a novel approach which can effectively improve the performance of PrLMs in text classification. A rule-based simplification process is applied to a given sentence. PrLMs are encouraged to predict the real label of the given sentence with auxiliary inputs from the simplified version. Using strong PrLMs (BERT and ELECTRA) as baselines, our approach can still further improve the performance in various text classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityGeneral ClassificationLanguage ModelingLanguage ModellingLexical SimplificationmodelSentencetext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Enhancing Robustness of Pre-trained Language Model with Lexical Simplification
For both human readers and pre-trained language models (PrLMs), lexical diversity may lead to confusion and inaccuracy when understanding the underlying semantic meanings of given sentences. By substituting complex words…
DiversityLanguage ModelingLanguage ModellingLexical Simplification+3Word Complexity Estimation for Japanese Lexical Simplification
We introduce three language resources for Japanese lexical simplification: 1) a large-scale word complexity lexicon, 2) the first synonym lexicon for converting complex words to simpler ones, and 3) the first toolkit for…
BenchmarkingLexical SimplificationMANTIS at TSAR-2022 Shared Task: Improved Unsupervised Lexical Simplification with Pretrained Encoders
In this paper we present our contribution to the TSAR-2022 Shared Task on Lexical Simplification of the EMNLP 2022 Workshop on Text Simplification, Accessibility, and Readability. Our approach builds on and extends the u…
Language ModelingLanguage ModellingLexical SimplificationNatural Language Inference+3Acquisition of Syntactic Simplification Rules for French
Text simplification is the process of reducing the lexical and syntactic complexity of a text while attempting to preserve (most of) its information content. It has recently emerged as an important research area, which h…
Information RetrievalMachine TranslationSentence CompressionText Simplification+1Automatic Lexical Simplification for Turkish
In this paper, we present the first automatic lexical simplification system for the Turkish language. Recent text simplification efforts rely on manually crafted simplified corpora and comprehensive NLP tools that can an…
Lexical SimplificationSentenceText Simplification