Integrating Semantic Knowledge into Lexical Embeddings Based on Information Content Measurement
Distributional word representations are widely used in NLP tasks. These representations are based on an assumption that words with a similar context tend to have a similar meaning. To improve the quality of the context-based embeddings, many researches have explored how to make full use of existing lexical resources. In this paper, we argue that while we incorporate the prior knowledge with context-based embeddings, words with different occurrences should be treated differently. Therefore, we propose to rely on the measurement of information content to control the degree of applying prior knowledge into context-based embeddings - different words would have different learning rates when adjusting their embeddings. In the result, we demonstrate that our embeddings get significant improvements on two different tasks: Word Similarity and Analogical Reasoning.
Code (1)
Tasks
Document ClassificationInformation RetrievalQuestion AnsweringWord EmbeddingsWord SimilaritySimilar Papers 제목 키워드 기반
Integrating Lexical Knowledge in Word Embeddings using Sprinkling and Retrofitting
Neural network based word embeddings, such as Word2Vec and GloVe, are purely data driven in that they capture the distributional information about words from the training corpus. Past works have attempted to improve thes…
Word EmbeddingsLexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution
Lexical substitution is the task of generating meaningful substitutes for a word in a given textual context. Contextual word embedding models have achieved state-of-the-art results in the lexical substitution task by rel…
SentenceSentence SimilarityIntegrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings
Models of acoustic word embeddings (AWEs) learn to map variable-length spoken word segments onto fixed-dimensionality vector representations such that different acoustic exemplars of the same word are projected nearby in…
FormMulti-Task LearningWord EmbeddingsIntegrating a Phrase Structure Corpus Grammar and a Lexical-Semantic Network: the HOLINET Knowledge Graph
In this paper we address the question of how to integrate grammar and lexical-semantic knowledge within a single and homogeneous knowledge graph. We introduce a graph modelling of grammar knowledge which enables its merg…
POSLexical Chains meet Word Embeddings in Document-level Statistical Machine Translation
Currently under review for EMNLP 2017 The phrase-based Statistical Machine Translation (SMT) approach deals with sentences in isolation, making it difficult to consider discourse context in translation. This poses a chal…
DecoderDocument SummarizationInformation RetrievalMachine Translation+4