Learning Word Embeddings without Context Vectors
Most word embedding algorithms such as word2vec or fastText construct two sort of vectors: for words and for contexts. Naive use of vectors of only one sort leads to poor results. We suggest using indefinite inner product in skip-gram negative sampling algorithm. This allows us to use only one sort of vectors without loss of quality. Our {``}context-free{''} cf algorithm performs on par with SGNS on word similarity datasets
Code (0)
등록된 구현이 없습니다.
Tasks
Learning Word EmbeddingsWord EmbeddingsWord SimilarityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Context-Aware Cross-Lingual Mapping
Cross-lingual word vectors are typically obtained by fitting an orthogonal matrix that maps the entries of a bilingual dictionary from a source to a target vector space. Word vectors, however, are most commonly used for …
RetrievalSentenceSentence EmbeddingsTranslation+2Context Vectors are Reflections of Word Vectors in Half the Dimensions
This paper takes a step towards theoretical analysis of the relationship between word embeddings and context embeddings in models such as word2vec. We start from basic probabilistic assumptions on the nature of word vect…
Text GenerationWord EmbeddingsDeriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection
One of the long-standing challenges in lexical semantics consists in learning representations of words which reflect their semantic properties. The remarkable success of word embeddings for this purpose suggests that hig…
SentenceWord EmbeddingsCWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling
Most existing topic models rely on bag-of-words (BOW) representation, which limits their ability to capture word order information and leads to challenges with out-of-vocabulary (OOV) words in new documents. Contextualiz…
Document ClassificationLanguage ModellingNatural Language UnderstandingNER+3Modelling General Properties of Nouns by Selectively Averaging Contextualised Embeddings
While the success of pre-trained language models has largely eliminated the need for high-quality static word vectors in many NLP applications, such vectors continue to play an important role in tasks where words need to…
Knowledge Base Completion