Geometry of Compositionality
This paper proposes a simple test for compositionality (i.e., literal usage) of a word or phrase in a context-specific way. The test is computationally simple, relying on no external resources and only uses a set of trained word vectors. Experiments show that the proposed method is competitive with state of the art and displays high accuracy in context-specific compositionality detection of a variety of natural language phenomena (idiomaticity, sarcasm, metaphor) for different datasets in multiple languages. The key insight is to connect compositionality to a curious geometric property of word embeddings, which is of independent interest.
Code (1)
Tasks
Word EmbeddingsSimilar Papers 제목 키워드 기반
Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models
Vision-Language Models (VLMs) learn a shared feature space for text and images, enabling the comparison of inputs of different modalities. While prior works demonstrated that VLMs organize natural language representation…
HairCUP: Hair Compositional Universal Prior for 3D Gaussian Avatars
We present a universal prior model for 3D head avatars with explicit hair compositionality. Existing approaches to build generalizable priors for 3D head avatars often adopt a holistic modeling approach, treating the fac…
Syntax-guided Neural Module Distillation to Probe Compositionality in Sentence Embeddings
Past work probing compositionality in sentence embedding models faces issues determining the causal impact of implicit syntax representations. Given a sentence, we construct a neural module net based on its syntax parse …
Semantic CompositionSentenceSentence EmbeddingSentence-Embedding+1Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder
Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, compositionality, and generalisation capabilities of Transformer-based auto…
Representation LearningInvestigating More Explainable and Partition-Free Compositionality Estimation for LLMs: A Rule-Generation Perspective
Compositional generalization tests are often used to estimate the compositionality of LLMs. However, such tests have the following limitations: (1) they only focus on the output results without considering LLMs' understa…