Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations
In this paper, we demonstrate that by utilizing sparse word representations, it becomes possible to surpass the results of more complex task-specific models on the task of fine-grained all-words word sense disambiguation. Our proposed algorithm relies on an overcomplete set of semantic basis vectors that allows us to obtain sparse contextualized word representations. We introduce such an information theory-inspired synset representation based on the co-occurrence of word senses and non-zero coordinates for word forms which allows us to achieve an aggregated F-score of 78.8 over a combination of five standard word sense disambiguating benchmark datasets. We also demonstrate the general applicability of our proposed framework by evaluating it towards part-of-speech tagging on four different treebanks. Our results indicate a significant improvement over the application of the dense word representations.
Code (0)
등록된 구현이 없습니다.
Tasks
Part-Of-Speech TaggingWord Sense DisambiguationSimilar Papers 제목 키워드 기반
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
Most supervised word sense disambiguation (WSD) systems build word-specific classifiers by leveraging labeled data. However, when using word-specific classifiers, the sparseness of annotations leads to inferior sense dis…
AllWord EmbeddingsWord Sense DisambiguationAn Unsupervised Word Sense Disambiguation System for Under-Resourced Languages
In this paper, we present Watasense, an unsupervised system for word sense disambiguation. Given a sentence, the system chooses the most relevant sense of each input word with respect to the semantic similarity between t…
Semantic SimilaritySemantic Textual SimilaritySentenceWord Sense DisambiguationUnsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation
Interpretability of a predictive model is a powerful feature that gains the trust of users in the correctness of the predictions. In word sense disambiguation (WSD), knowledge-based systems tend to be much more interpret…
Word Sense DisambiguationUsing Linked Disambiguated Distributional Networks for Word Sense Disambiguation
We introduce a new method for unsupervised knowledge-based word sense disambiguation (WSD) based on a resource that links two types of sense-aware lexical networks: one is induced from a corpus using distributional seman…
Machine TranslationTranslationWord EmbeddingsWord Sense DisambiguationLanguage Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation
Contextual embeddings represent a new generation of semantic representations learned from Neural Language Modelling (NLM) that addresses the issue of meaning conflation hampering traditional word embeddings. In this work…
Language ModellingLEMMAWord EmbeddingsWord Sense Disambiguation