A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
Lexical substitution in context is an extremely powerful technology that can be used as a backbone of various NLP applications, such as word sense induction, lexical relation extraction, data augmentation, etc. In this paper, we present a large-scale comparative study of popular neural language and masked language models (LMs and MLMs), such as context2vec, ELMo, BERT, XLNet, applied to the task of lexical substitution. We show that already competitive results achieved by SOTA LMs/MLMs can be further improved if information about the target word is injected properly, and compare several target injection methods. In addition, we provide analysis of the types of semantic relations between the target and substitutes generated by different models providing insights into what kind of words are really generated or given by annotators as substitutes.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationRelation ExtractionWord Sense InductionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution
Lexical substitution, i.e. generation of plausible words that can replace a particular target word in a given context, is an extremely powerful technology that can be used as a backbone of various NLP applications, inclu…
Data AugmentationRelation ExtractionWord Sense InductionA comparative study of word embeddings and other features for lexical complexity detection in French
Lexical complexity detection is an important step for automatic text simplification which serves to make informed lexical substitutions. In this study, we experiment with word embeddings for measuring the complexity of F…
Text SimplificationWord EmbeddingsA Preliminary Study of Croatian Lexical Substitution
Lexical substitution is a task of determining a meaning-preserving replacement for a word in context. We report on a preliminary study of this task for the Croatian language on a small-scale lexical sample dataset, manua…
Information RetrievalMachine TranslationWord Sense DisambiguationMetaheuristic Approaches to Lexical Substitution and Simplification
In this paper, we propose using metaheuristics{---}in particular, simulated annealing and the new D-Bees algorithm{---}to solve word sense disambiguation as an optimization problem within a knowledge-based lexical substi…
Lexical SimplificationMachine TranslationQuestion AnsweringSentence Compression+3Contextualized context2vec
Lexical substitution ranks substitution candidates from the viewpoint of paraphrasability for a target word in a given sentence. There are two major approaches for lexical substitution: (1) generating contextualized word…
SentenceWord Embeddings