Analyzing Structures in the Semantic Vector Space: A Framework for Decomposing Word Embeddings
Word embeddings are rich word representations, which in combination with deep neural networks, lead to large performance gains for many NLP tasks. However, word embeddings are represented by dense, real-valued vectors and they are therefore not directly interpretable. Thus, computational operations based on them are also not well understood. In this paper, we present an approach for analyzing structures in the semantic vector space to get a better understanding of the underlying semantic encoding principles. We present a framework for decomposing word embeddings into smaller meaningful units which we call sub-vectors. The framework opens up a wide range of possibilities analyzing phenomena in vector space semantics, as well as solving concrete NLP problems: We introduce the category completion task and show that a sub-vector based approach is superior to supervised techniques; We present a sub-vector based method for solving the word analogy task, which substantially outperforms different variants of the traditional vector-offset method.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces
Embedding spaces are fundamental to modern AI, translating raw data into high-dimensional vectors that encode rich semantic relationships. Yet, their internal structures remain opaque, with existing approaches often sacr…
Bias DetectionTracking Temporal Dynamics of Vector Sets with Gaussian Process
Understanding the temporal evolution of sets of vectors is a fundamental challenge across various domains, including ecology, crime analysis, and linguistics. For instance, ecosystem structures evolve due to interactions…
Gaussian ProcessesExploring Scholarly Data by Semantic Query on Knowledge Graph Embedding Space
The trends of open science have enabled several open scholarly datasets which include millions of papers and authors. Managing, exploring, and utilizing such large and complicated datasets effectively are challenging. In…
Graph EmbeddingKnowledge Graph CompletionKnowledge Graph EmbeddingKnowledge Graphs+3Bag-of-Vector Embeddings of Dependency Graphs for Semantic Induction
Vector-space models, from word embeddings to neural network parsers, have many advantages for NLP. But how to generalise from fixed-length word vectors to a vector space for arbitrary linguistic structures is still uncle…
Natural Language InferenceSemantic Textual SimilarityWord EmbeddingsLearning to Embed Semantic Correspondence for Natural Language Understanding
While learning embedding models has yielded fruitful results in several NLP subfields, most notably Word2Vec, embedding correspondence has relatively not been well explored especially in the context of natural language u…
Natural Language UnderstandingRe-RankingSemantic correspondenceSlot Filling