Word Sense Disambiguation with Transformer Models
In this paper, we tackle the task of Word Sense Disambiguation (WSD). We present our system submitted to the Word-in-Context Target Sense Verification challenge, part of the SemDeep workshop at IJCAI 2020 (Breit et al., 2020). That challenge asks participants to predict if a specific mention of a word in a text matches a pre-defined sense. Our approach uses pre-trained transformer models such as BERT that are fine-tuned on the task using different architecture strategies. Our model achieves the best accuracy and precision on Subtask 1 – make use of definitions for deciding whether the target word in context corresponds to the given sense or not. We believe the strategies we explored in the context of this challenge can be useful to other Natural Language Processing tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Entity LinkingWord Sense DisambiguationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ChiSense-12: An English Sense-Annotated Child-Directed Speech Corpus
Language acquisition research has benefitted from the use of annotated corpora of child-directed speech to examine key questions about how children learn and process language in real-world contexts. However, a lack of se…
Language AcquisitionWord Sense DisambiguationTraining Bi-Encoders for Word Sense Disambiguation
Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning the correct sense of a word in a given context, is no exception. St…
Word Sense DisambiguationQuasi Bidirectional Encoder Representations from Transformers for Word Sense Disambiguation
While contextualized embeddings have produced performance breakthroughs in many Natural Language Processing (NLP) tasks, Word Sense Disambiguation (WSD) has not benefited from them yet. In this paper, we introduce QBERT,…
Word Sense DisambiguationConSeC: Word Sense Disambiguation as Continuous Sense Comprehension
Supervised systems have nowadays become the standard recipe for Word Sense Disambiguation (WSD), with Transformer-based language models as their primary ingredient. However, while these systems have certainly attained un…
Word Sense DisambiguationA Comparative Study of Transformers on Word Sense Disambiguation
Recent years of research in Natural Language Processing (NLP) have witnessed dramatic growth in training large models for generating context-aware language representations. In this regard, numerous NLP systems have lever…
Word EmbeddingsWord Sense Disambiguation