A Differentiable Self-disambiguated Sense Embedding Model via Scaled Gumbel Softmax
We present a differentiable multi-prototype word representation model that disentangles senses of polysemous words and produces meaningful sense-specific embeddings without external resources. It jointly learns how to disambiguate senses given local context and how to represent senses using hard attention. Unlike previous multi-prototype models, our model approximates discrete sense selection in a differentiable manner via a modified Gumbel softmax. We also propose a novel human evaluation task that quantitatively measures (1) how meaningful the learned sense groups are to humans and (2) how well the model is able to disambiguate senses given a context sentence. Our model outperforms competing approaches on both human evaluations and multiple word similarity tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Hard AttentionSentenceWord SimilaritySimilar Papers 제목 키워드 기반
Disambiguated skip-gram model
We present disambiguated skip-gram: a neural-probabilistic model for learning multi-sense distributed representations of words. Disambiguated skip-gram jointly estimates a skip-gram-like context word prediction model and…
Image CaptioningmodelWord EmbeddingsWord Sense Disambiguation+1Vector representations of text data in deep learning
In this dissertation we report results of our research on dense distributed representations of text data. We propose two novel neural models for learning such representations. The first model learns representations at th…
Deep LearningInformation RetrievalPart-Of-Speech TaggingRetrieval+3sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings
Neural word representations have proven useful in Natural Language Processing (NLP) tasks due to their ability to efficiently model complex semantic and syntactic word relationships. However, most techniques model only o…
Dependency ParsingWord EmbeddingsWord Sense DisambiguationInducing and Embedding Senses with Scaled Gumbel Softmax
Methods for learning word sense embeddings represent a single word with multiple sense-specific vectors. These methods should not only produce interpretable sense embeddings, but should also learn how to select which sen…
Moving Down the Long Tail of Word Sense Disambiguation with Gloss-Informed Biencoders
A major obstacle in Word Sense Disambiguation (WSD) is that word senses are not uniformly distributed, causing existing models to generally perform poorly on senses that are either rare or unseen during training. We prop…
Word Sense Disambiguation