Enhancing Domain Word Embedding via Latent Semantic Imputation
We present a novel method named Latent Semantic Imputation (LSI) to transfer external knowledge into semantic space for enhancing word embedding. The method integrates graph theory to extract the latent manifold structure of the entities in the affinity space and leverages non-negative least squares with standard simplex constraints and power iteration method to derive spectral embeddings. It provides an effective and efficient approach to combining entity representations defined in different Euclidean spaces. Specifically, our approach generates and imputes reliable embedding vectors for low-frequency words in the semantic space and benefits downstream language tasks that depend on word embedding. We conduct comprehensive experiments on a carefully designed classification problem and language modeling and demonstrate the superiority of the enhanced embedding via LSI over several well-known benchmark embeddings. We also confirm the consistency of the results under different parameter settings of our method.
Code (1)
Tasks
ImputationLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation
The most interesting words in scientific texts will often be novel or rare. This presents a challenge for scientific word embedding models to determine quality embedding vectors for useful terms that are infrequent or ne…
ImputationKnowledge GraphsWord EmbeddingsLatent Topic Embedding
Topic modeling and word embedding are two important techniques for deriving latent semantics from data. General-purpose topic models typically work in coarse granularity by capturing word co-occurrence at the document/se…
SentenceTopic ModelsWord EmbeddingsNeutralizing Gender Bias in Word Embeddings with Latent Disentanglement and Counterfactual Generation
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstre…
counterfactualDisentanglementWord EmbeddingsNeutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstre…
counterfactualDisentanglementSentiment AnalysisWord EmbeddingsInterpretable Word Embeddings via Informative Priors
Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital h…
Word Embeddings