Conditional entropy minimization principle for learning domain invariant representation features
Invariance-principle-based methods such as Invariant Risk Minimization (IRM), have recently emerged as promising approaches for Domain Generalization (DG). Despite promising theory, such approaches fail in common classification tasks due to the mixing of true invariant features and spurious invariant features. To address this, we propose a framework based on the conditional entropy minimization (CEM) principle to filter-out the spurious invariant features leading to a new algorithm with a better generalization capability. We show that our proposed approach is closely related to the well-known Information Bottleneck (IB) framework and prove that under certain assumptions, entropy minimization can exactly recover the true invariant features. Our approach provides competitive classification accuracy compared to recent theoretically-principled state-of-the-art alternatives across several DG datasets.
Code (2)
Tasks
Domain GeneralizationSimilar Papers 제목 키워드 기반
Learning Invariant Reward Functions through Trajectory Interventions
Inverse reinforcement learning methods aim to retrieve the reward function of a Markov decision process based on a dataset of expert demonstrations. The commonplace scarcity of such demonstrations potentially leads to th…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Domain Generalization via Entropy Regularization
Domain generalization aims to learn from multiple source domains a predictive model that can generalize to unseen target domains. One essential problem in domain generalization is to learn discriminative domain-invariant…
Domain GeneralizationCausal Domain Adaptation with Copula Entropy based Conditional Independence Test
Domain Adaptation (DA) is a typical problem in machine learning that aims to transfer the model trained on source domain to target domain with different distribution. Causal DA is a special case of DA that solves the pro…
Causal DiscoveryDomain AdaptationRegularized Conditional Alignment for Multi-Domain Text Classification
The most successful multi-domain text classification (MDTC) approaches employ the shared-private paradigm to facilitate the enhancement of domain-invariant features through domain-specific attributes. Additionally, they …
Classificationtext-classificationText ClassificationSwap distance minimization beyond entropy minimization in word order variation
Here we consider the problem of all the possible orders of a linguistic structure formed by $n$ elements, for instance, subject, direct object and verb ($n=3$) or subject, direct object, indirect object and verb ($n=4$).…