Correlation-based Intrinsic Evaluation of Word Vector Representations
We introduce QVEC-CCA--an intrinsic evaluation metric for word vector representations based on correlations of learned vectors with features extracted from linguistic resources. We show that QVEC-CCA scores are an effective proxy for a range of extrinsic semantic and syntactic tasks. We also show that the proposed evaluation obtains higher and more consistent correlations with downstream tasks, compared to existing approaches to intrinsic evaluation of word vectors that are based on word similarity.
Code (0)
등록된 구현이 없습니다.
Tasks
Word SimilaritySimilar Papers 제목 키워드 기반
Intrinsic Subspace Evaluation of Word Embedding Representations
We introduce a new methodology for intrinsic evaluation of word representations. Specifically, we identify four fundamental criteria based on the characteristics of natural language that pose difficulties to NLP systems;…
What's in Your Embedding, And How It Predicts Task Performance
Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. We present a new approach based on scaled-up qualitative analysis of word vector neighborho…
Named Entity Recognition (NER)Semantic Role LabelingWord EmbeddingsFind the word that does not belong: A Framework for an Intrinsic Evaluation of Word Vector Representations
Problems With Evaluation of Word Embeddings Using Word Similarity Tasks
Lacking standardized extrinsic evaluation methods for vector representations of words, the NLP community has relied heavily on word similarity tasks as a proxy for intrinsic evaluation of word vectors. Word similarity ev…
Semantic SimilaritySemantic Textual SimilarityWord EmbeddingsWord SimilarityHypothesis Testing based Intrinsic Evaluation of Word Embeddings
We introduce the cross-match test - an exact, distribution free, high-dimensional hypothesis test as an intrinsic evaluation metric for word embeddings. We show that cross-match is an effective means of measuring distrib…
Machine TranslationTranslationTwo-sample testingWord Embeddings