Training Bi-Encoders for Word Sense Disambiguation
Modern transformer-based neural architectures yield impressive results in nearly every NLP task and Word Sense Disambiguation, the problem of discerning the correct sense of a word in a given context, is no exception. State-of-the-art approaches in WSD today leverage lexical information along with pre-trained embeddings from these models to achieve results comparable to human inter-annotator agreement on standard evaluation benchmarks. In the same vein, we experiment with several strategies to optimize bi-encoders for this specific task and propose alternative methods of presenting lexical information to our model. Through our multi-stage pre-training and fine-tuning pipeline we further the state of the art in Word Sense Disambiguation.
Code (0)
등록된 구현이 없습니다.
Tasks
Word Sense DisambiguationSimilar Papers 제목 키워드 기반
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We prop…
Word Sense DisambiguationMoving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We prop…
Word Sense DisambiguationEncoders Help You Disambiguate Word Senses in Neural Machine Translation
Neural machine translation (NMT) has achieved new state-of-the-art performance in translating ambiguous words. However, it is still unclear which component dominates the process of disambiguation. In this paper, we explo…
DecoderMachine TranslationNMTTranslation+1Zero-shot Word Sense Disambiguation using Sense Definition Embeddings
Word Sense Disambiguation (WSD) is a long-standing but open problem in Natural Language Processing (NLP). WSD corpora are typically small in size, owing to an expensive annotation process. Current supervised WSD methods …
Generalized Zero-Shot LearningGraph EmbeddingKnowledge Graph EmbeddingSentence+2MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation
Recent approaches to word sense disambiguation (WSD) utilize encodings of the sense gloss (definition), in addition to the input context, to improve performance. In this work we demonstrate that this approach can be adap…
Word Sense Disambiguation