Understanding Higher-Order Correlations Among Semantic Components in Embeddings
Independent Component Analysis (ICA) offers interpretable semantic components of embeddings. While ICA theory assumes that embeddings can be linearly decomposed into independent components, real-world data often do not satisfy this assumption. Consequently, non-independencies remain between the estimated components, which ICA cannot eliminate. We quantified these non-independencies using higher-order correlations and demonstrated that when the higher-order correlation between two components is large, it indicates a strong semantic association between them, along with many words sharing common meanings with both components. The entire structure of non-independencies was visualized using a maximum spanning tree of semantic components. These findings provide deeper insights into embeddings through ICA.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Implicit Semantic-Aware Communication Based on Hypergraph Reasoning
Semantic-aware communication has emerged as a transformative paradigm for next-generation communication systems, shifting the fundamental goal from transmitting bit-level symbols to reliably recovering and understanding …
Graph EmbeddingLabel Distribution Amendment with Emotional Semantic Correlations for Facial Expression Recognition
By utilizing label distribution learning, a probability distribution is assigned for a facial image to express a compound emotion, which effectively improves the problem of label uncertainties and noises occurred in one-…
Facial Expression RecognitionFacial Expression Recognition (FER)Graph Structured Prediction Energy Networks
For joint inference over multiple variables, a variety of structured prediction techniques have been developed to model correlations among variables and thereby improve predictions. However, many classical approaches suf…
PredictionStructured PredictionVSFormer: Mining Correlations in Flexible View Set for Multi-view 3D Shape Understanding
View-based methods have demonstrated promising performance in 3D shape understanding. However, they tend to make strong assumptions about the relations between views or learn the multi-view correlations indirectly, which…
Higher Order Correlation Analysis for Multi-View Learning
Multi-view learning is frequently used in data science. The pairwise correlation maximization is a classical approach for exploring the consensus of multiple views. Since the pairwise correlation is inherent for two view…
MULTI-VIEW LEARNING