Dependence Induced Representations

2024-11-22 · Xiangxiang Xu, Lizhong Zheng

We study the problem of learning feature representations from a pair of random variables, where we focus on the representations that are induced by their dependence. We provide sufficient and necessary conditions for such dependence induced representations, and illustrate their connections to Hirschfeld--Gebelein--R\'{e}nyi (HGR) maximal correlation functions and minimal sufficient statistics. We characterize a large family of loss functions that can learn dependence induced representations, including cross entropy, hinge loss, and their regularized variants. In particular, we show that the features learned from this family can be expressed as the composition of a loss-dependent function and the maximal correlation function, which reveals a key connection between representations learned from different losses. Our development also gives a statistical interpretation of the neural collapse phenomenon observed in deep classifiers. Finally, we present the learning design based on the feature separation, which allows hyperparameter tuning during inference.

📄 PDF Abstract BibTeX arXiv:2411.15328

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Disentanglement by means of action-induced representations

2026-02-06 · Gorka Muñoz-Gil, Hendrik Poulsen Nautrup, Arunava Majumder, Paulin de Schoulepnikoff 외 arxiv

Learning interpretable representations with variational autoencoders (VAEs) is a major goal of representation learning. The main challenge lies in obtaining disentangled representations, where each latent dimension corre…

Representation Learning

Incorporating Interventional Independence Improves Robustness against Interventional Distribution Shift

2025-07-07 · Gautam Sreekumar, Vishnu Naresh Boddeti arxiv

We study the problem of learning robust discriminative representations of causally related latent variables given the underlying causal graph and a training set comprising passively collected observational data and inter…

Facial Attribute Classification

Hyperprior Induced Unsupervised Disentanglement of Latent Representations

2018-09-12 · Abdul Fatir Ansari, Harold Soh

We address the problem of unsupervised disentanglement of latent representations learnt via deep generative models. In contrast to current approaches that operate on the evidence lower bound (ELBO), we argue that statist…

Disentanglement

Conditional independence structures over four discrete random variables revisited: conditional Ingleton inequalities

2020-12-07 · Milan Studeny

The paper deals with conditional linear information inequalities valid for entropy functions induced by discrete random variables. Specifically, the so-called conditional Ingleton inequalities are in the center of intere…

valid

Diversity-Induced Multi-View Subspace Clustering

2015-06-01 · CVPR 2015 6 · Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu 외

In this paper, we focus on how to boost the multi-view clustering by exploring the complementary information among multi-view features. A multi-view clustering framework, called Diversity-induced Multi-view Subspace Clus…

Clustering Diversity Face Clustering Multi-view Subspace Clustering