Non-Parametric Few-Shot Learning for Word Sense Disambiguation
Word sense disambiguation (WSD) is a long-standing problem in natural language processing. One significant challenge in supervised all-words WSD is to classify among senses for a majority of words that lie in the long-tail distribution. For instance, 84% of the annotated words have less than 10 examples in the SemCor training data. This issue is more pronounced as the imbalance occurs in both word and sense distributions. In this work, we propose MetricWSD, a non-parametric few-shot learning approach to mitigate this data imbalance issue. By learning to compute distances among the senses of a given word through episodic training, MetricWSD transfers knowledge (a learned metric space) from high-frequency words to infrequent ones. MetricWSD constructs the training episodes tailored to word frequencies and explicitly addresses the problem of the skewed distribution, as opposed to mixing all the words trained with parametric models in previous work. Without resorting to any lexical resources, MetricWSD obtains strong performance against parametric alternatives, achieving a 75.1 F1 score on the unified WSD evaluation benchmark (Raganato et al., 2017b). Our analysis further validates that infrequent words and senses enjoy significant improvement.
Code (1)
Tasks
Few-Shot LearningWord Sense DisambiguationSimilar Papers 제목 키워드 기반
Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives
Lately proposed Word Sense Disambiguation (WSD) systems have approached the estimated upper bound of the task on standard evaluation benchmarks. However, these systems typically implement the disambiguation of words in a…
SentenceWord Sense DisambiguationRare and Zero-shot Word Sense Disambiguation using Z-Reweighting
Word sense disambiguation (WSD) is a crucial problem in the natural language processing (NLP) community. Current methods achieve decent performance by utilizing supervised learning and large pre-trained language models. …
RelationWord Sense DisambiguationFEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary
Current models for Word Sense Disambiguation (WSD) struggle to disambiguate rare senses, despite reaching human performance on global WSD metrics. This stems from a lack of data for both modeling and evaluating rare sens…
Transfer LearningWord Sense DisambiguationTranslate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models
Pretrained Language Models (PLMs) learn rich cross-lingual knowledge and can be finetuned to perform well on diverse tasks such as translation and multilingual word sense disambiguation (WSD). However, they often struggl…
TranslationWord Sense DisambiguationMeta-Learning with Variational Semantic Memory for Word Sense Disambiguation
A critical challenge faced by supervised word sense disambiguation (WSD) is the lack of large annotated datasets with sufficient coverage of words in their diversity of senses. This inspired recent research on few-shot W…
DiversityMeta-LearningVariational InferenceWord Sense Disambiguation