Adapting BERT for Word Sense Disambiguation with Gloss Selection Objective and Example Sentences
Domain adaptation or transfer learning using pre-trained language models such as BERT has proven to be an effective approach for many natural language processing tasks. In this work, we propose to formulate word sense disambiguation as a relevance ranking task, and fine-tune BERT on sequence-pair ranking task to select the most probable sense definition given a context sentence and a list of candidate sense definitions. We also introduce a data augmentation technique for WSD using existing example sentences from WordNet. Using the proposed training objective and data augmentation technique, our models are able to achieve state-of-the-art results on the English all-words benchmark datasets.
Code (1)
Tasks
Data AugmentationDomain AdaptationSentenceTransfer LearningWord Sense DisambiguationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge
Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. Traditional supervised methods rarely take into consideration the lexical resources like WordNet, which are widel…
Word Sense DisambiguationBuilding on Huang et al. GlossBERT for Word Sense Disambiguation
We propose to take on the problem ofWord Sense Disambiguation (WSD). In language, words of the same form can take different meanings depending on context. While humans easily infer the meaning or gloss of such words by t…
Word Sense DisambiguationImproved Word Sense Disambiguation with Enhanced Sense Representations
Current state-of-the-art supervised word sense disambiguation (WSD) systems (such as GlossBERT and bi-encoder model) yield surprisingly good results by purely leveraging pre-trained language models and short dictionary d…
Word Sense DisambiguationContext-gloss Augmentation for Improving Word Sense Disambiguation
The goal of Word Sense Disambiguation (WSD) is to identify the sense of a polysemous word in a specific context. Deep-learning techniques using BERT have achieved very promising results in the field and different methods…
Data AugmentationSentenceTranslationWord Sense DisambiguationArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD
Using pre-trained transformer models such as BERT has proven to be effective in many NLP tasks. This paper presents our work to fine-tune BERT models for Arabic Word Sense Disambiguation (WSD). We treated the WSD task as…
Binary ClassificationSentenceWord Sense Disambiguation