Intrinsic Subspace Evaluation of Word Embedding Representations
We introduce a new methodology for intrinsic evaluation of word representations. Specifically, we identify four fundamental criteria based on the characteristics of natural language that pose difficulties to NLP systems; and develop tests that directly show whether or not representations contain the subspaces necessary to satisfy these criteria. Current intrinsic evaluations are mostly based on the overall similarity or full-space similarity of words and thus view vector representations as points. We show the limits of these point-based intrinsic evaluations. We apply our evaluation methodology to the comparison of a count vector model and several neural network models and demonstrate important properties of these models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Methodical Evaluation of Arabic Word Embeddings
Many unsupervised learning techniques have been proposed to obtain meaningful representations of words from text. In this study, we evaluate these various techniques when used to generate Arabic word embeddings. We first…
Document ClassificationLearning Word EmbeddingsNamed Entity Recognition (NER)Word EmbeddingsHypothesis Testing based Intrinsic Evaluation of Word Embeddings
We introduce the cross-match test - an exact, distribution free, high-dimensional hypothesis test as an intrinsic evaluation metric for word embeddings. We show that cross-match is an effective means of measuring distrib…
Machine TranslationTranslationTwo-sample testingWord EmbeddingsA Survey of Word Embeddings Evaluation Methods
Word embeddings are real-valued word representations able to capture lexical semantics and trained on natural language corpora. Models proposing these representations have gained popularity in the recent years, but the i…
Embeddings EvaluationSurveyWord EmbeddingsJust Rank: Rethinking Evaluation with Word and Sentence Similarities
Word and sentence embeddings are useful feature representations in natural language processing. However, intrinsic evaluation for embeddings lags far behind, and there has been no significant update since the past decade…
BenchmarkingSemantic SimilaritySemantic Textual SimilaritySentence+4Word Embedding based New Corpus for Low-resourced Language: Sindhi
Representing words and phrases into dense vectors of real numbers which encode semantic and syntactic properties is a vital constituent in natural language processing (NLP). The success of neural network (NN) models in N…
Word Embeddings