Analyzing Word Embedding Through Structural Equation Modeling
Many researchers have tried to predict the accuracies of extrinsic evaluation by using intrinsic evaluation to evaluate word embedding. The relationship between intrinsic and extrinsic evaluation, however, has only been studied with simple correlation analysis, which has difficulty capturing complex cause-effect relationships and integrating external factors such as the hyperparameters of word embedding. To tackle this problem, we employ partial least squares path modeling (PLS-PM), a method of structural equation modeling developed for causal analysis. We propose a causal diagram consisting of the evaluation results on the BATS, VecEval, and SentEval datasets, with a causal hypothesis that linguistic knowledge encoded in word embedding contributes to solving downstream tasks. Our PLS-PM models are estimated with 600 word embeddings, and we prove the existence of causal relations between linguistic knowledge evaluated on BATS and the accuracies of downstream tasks evaluated on VecEval and SentEval in our PLS-PM models. Moreover, we show that the PLS-PM models are useful for analyzing the effect of hyperparameters, including the training algorithm, corpus, dimension, and context window, and for validating the effectiveness of intrinsic evaluation.
Code (0)
등록된 구현이 없습니다.
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
What's in an Embedding? Analyzing Word Embeddings through Multilingual Evaluation
Equation Embeddings
We present an unsupervised approach for discovering semantic representations of mathematical equations. Equations are challenging to analyze because each is unique, or nearly unique. Our method, which we call equation em…
ArticlesSemantic SimilaritySemantic Textual SimilarityWord Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsWord Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsAnalyzing Continuous Semantic Shifts with Diachronic Word Similarity Matrices
The meanings and relationships of words shift over time. This phenomenon is referred to as semantic shift. Research focused on understanding how semantic shifts occur over multiple time periods is essential for gaining a…
Word EmbeddingsWord Similarity