Paraphrases do not explain word analogies
Many types of distributional word embeddings (weakly) encode linguistic regularities as directions (the difference between "jump" and "jumped" will be in a similar direction to that of "walk" and "walked," and so on). Several attempts have been made to explain this fact. We respond to Allen and Hospedales' recent (ICML, 2019) theoretical explanation, which claims that word2vec and GloVe will encode linguistic regularities whenever a specific relation of paraphrase holds between the four words involved in the regularity. We demonstrate that the explanation does not go through: the paraphrase relations needed under this explanation do not hold empirically.
Code (1)
Tasks
Word EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Temporal Word Analogies: Identifying Lexical Replacement with Diachronic Word Embeddings
This paper introduces the concept of temporal word analogies: pairs of words which occupy the same semantic space at different points in time. One well-known property of word embeddings is that they are able to effective…
ArticlesDiachronic Word EmbeddingsWord EmbeddingsFuzzy paraphrases in learning word representations with a lexicon
A synonym of a polysemous word is usually only the paraphrase of one sense among many. When lexicons are used to improve vector-space word representations, such paraphrases are unreliable and bring noise to the vector-sp…
Paraphrases as Foreign Languages in Multilingual Neural Machine Translation
Paraphrases, the rewordings of the same semantic meaning, are useful for improving generalization and translation. However, prior works only explore paraphrases at the word or phrase level, not at the sentence or corpus …
Machine TranslationNMTSentenceTAG+2ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models
Over the past decade, analogies, in the form of word-level analogies, have played a significant role as an intrinsic measure of evaluating the quality of word embedding methods such as word2vec. Modern large language mod…
NegationSentencePortuguese Word Embeddings: Evaluating on Word Analogies and Natural Language Tasks
Word embeddings have been found to provide meaningful representations for words in an efficient way; therefore, they have become common in Natural Language Processing sys- tems. In this paper, we evaluated different word…
POSPOS TaggingSemantic SimilaritySemantic Textual Similarity+2