Learning and Evaluating Sparse Interpretable Sentence Embeddings
Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have the benefit of increased interpretability properties: to some degree, each dimension can be understood by a human and associated with a recognizable feature in the data. In this paper, we transfer this idea to sentence embeddings and explore several approaches to obtain a sparse representation. We further introduce a novel, quantitative and automated evaluation metric for sentence embedding interpretability, based on topic coherence methods. We observe an increase in interpretability compared to dense models, on a dataset of movie dialogs and on the scene descriptions from the MS COCO dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
Finding Meaning in Embeddings: Concept Separation Curves
Sentence embedding techniques aim to encode key concepts of a sentence's meaning in a vector space. However, the majority of evaluation approaches for sentence embedding quality rely on the use of additional classifiers …
Word Equations: Inherently Interpretable Sparse Word Embeddingsthrough Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsWord Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding
Word embeddings are a powerful natural language processing technique, but they are extremely difficult to interpret. To enable interpretable NLP models, we create vectors where each dimension is inherently interpretable.…
Word EmbeddingsAligning Sentence Embeddings to Human Concepts via Sparse Autoencoders
Dense sentence embeddings are fundamental to modern Retrieval-Augmented Generation (RAG) systems but suffer from a lack of interpretability due to feature superposition. This opacity hinders the alignment of retrieval pr…
Information RetrievalSparseGAN: Sparse Generative Adversarial Network for Text Generation
It is still a challenging task to learn a neural text generation model under the framework of generative adversarial networks (GANs) since the entire training process is not differentiable. The existing training strategi…
Generative Adversarial NetworkSentenceText GenerationWord Embeddings