Word2Sense: Sparse Interpretable Word Embeddings
We present an unsupervised method to generate Word2Sense word embeddings that are interpretable {---} each dimension of the embedding space corresponds to a fine-grained sense, and the non-negative value of the embedding along the j-th dimension represents the relevance of the j-th sense to the word. The underlying LDA-based generative model can be extended to refine the representation of a polysemous word in a short context, allowing us to use the embedings in contextual tasks. On computational NLP tasks, Word2Sense embeddings compare well with other word embeddings generated by unsupervised methods. Across tasks such as word similarity, entailment, sense induction, and contextual interpretation, Word2Sense is competitive with the state-of-the-art method for that task. Word2Sense embeddings are at least as sparse and fast to compute as prior art.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings
Adding interpretability to word embeddings represents an area of active research in text representation. Recent work has explored thepotential of embedding words via so-called polar dimensions (e.g. good vs. bad, correct…
Word EmbeddingsWord Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsWord Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsUnsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation
The current trend in NLP is the use of highly opaque models, e.g. neural networks and word embeddings. While these models yield state-of-the-art results on a range of tasks, their drawback is poor interpretability. On th…
Word EmbeddingsWord Sense DisambiguationWord Sense InductionSLICE: Supersense-based Lightweight Interpretable Contextual Embeddings
Contextualised embeddings such as BERT have become de facto state-of-the-art references in many NLP applications, thanks to their impressive performances. However, their opaqueness makes it hard to interpret their behavi…