Hypothesis Testing based Intrinsic Evaluation of Word Embeddings
We introduce the cross-match test - an exact, distribution free, high-dimensional hypothesis test as an intrinsic evaluation metric for word embeddings. We show that cross-match is an effective means of measuring distributional similarity between different vector representations and of evaluating the statistical significance of different vector embedding models. Additionally, we find that cross-match can be used to provide a quantitative measure of linguistic similarity for selecting bridge languages for machine translation. We demonstrate that the results of the hypothesis test align with our expectations and note that the framework of two sample hypothesis testing is not limited to word embeddings and can be extended to all vector representations.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationTwo-sample testingWord EmbeddingsSimilar Papers 제목 키워드 기반
The Limitations of Cross-language Word Embeddings Evaluation
The aim of this work is to explore the possible limitations of existing methods of cross-language word embeddings evaluation, addressing the lack of correlation between intrinsic and extrinsic cross-language evaluation m…
Embeddings EvaluationWord EmbeddingsRobust and Consistent Estimation of Word Embedding for Bangla Language by fine-tuning Word2Vec Model
Word embedding or vector representation of word holds syntactical and semantic characteristics of a word which can be an informative feature for any machine learning-based models of natural language processing. There are…
Keyword ExtractionWord EmbeddingsWhat's in Your Embedding, And How It Predicts Task Performance
Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. We present a new approach based on scaled-up qualitative analysis of word vector neighborho…
Named Entity Recognition (NER)Semantic Role LabelingWord EmbeddingsAnalyzing Word Embedding Through Structural Equation Modeling
Many researchers have tried to predict the accuracies of extrinsic evaluation by using intrinsic evaluation to evaluate word embedding. The relationship between intrinsic and extrinsic evaluation, however, has only been …
Word EmbeddingsMeasuring Gender Bias in Word Embeddings across Domains and Discovering New Gender Bias Word Categories
Prior work has shown that word embeddings capture human stereotypes, including gender bias. However, there is a lack of studies testing the presence of specific gender bias categories in word embeddings across diverse do…
Bias DetectionClusteringGender Bias DetectionTwo-sample testing+1