Moving Down the Long Tail of Word Sense Disambiguation with Gloss Informed Bi-encoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We propose a bi-encoder model that independently embeds (1) the target word with its surrounding context and (2) the dictionary definition, or gloss, of each sense. The encoders are jointly optimized in the same representation space, so that sense disambiguation can be performed by finding the nearest sense embedding for each target word embedding. Our system outperforms previous state-of-the-art models on English all-words WSD; these gains predominantly come from improved performance on rare senses, leading to a 31.1{\%} error reduction on less frequent senses over prior work. This demonstrates that rare senses can be more effectively disambiguated by modeling their definitions.
Code (0)
등록된 구현이 없습니다.
Tasks
Word Sense DisambiguationSimilar Papers 제목 키워드 기반
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We prop…
Word Sense DisambiguationQuantum-inspired Representation for Long-tail Senses of Word Sense Disambiguation
Data imbalance, also known as the long-tailed distribution of data, is an important challenge for data-driven models. Due to the long tail phenomenon of word sense distribution in linguistics, it is difficult to learn ac…
Data AugmentationWord Sense DisambiguationBi-Matching Mechanism to Combat the Long Tail of Word Sense Disambiguation
The long tail phenomenon of word sense distribution in linguistics causes the Word Sense Disambiguation (WSD) task to face a serious polarization of word sense distribution, that is, Most Frequent Senses (MFSs) with huge…
Few-Shot LearningWord Sense DisambiguationNon-Parametric Few-Shot Learning for Word Sense Disambiguation
Word sense disambiguation (WSD) is a long-standing problem in natural language processing. One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-ta…
Few-Shot LearningWord Sense DisambiguationWord2Sense: Sparse Interpretable Word Embeddings
We present an unsupervised method to generate Word2Sense word embeddings that are interpretable {---} each dimension of the embedding space corresponds to a fine-grained sense, and the non-negative value of the embedding…
Word EmbeddingsWord Similarity