Learning Optimal Features via Partial Invariance

2023-01-28 · Moulik Choraria, Ibtihal Ferwana, Ankur Mani, Lav R. Varshney

Learning models that are robust to distribution shifts is a key concern in the context of their real-life applicability. Invariant Risk Minimization (IRM) is a popular framework that aims to learn robust models from multiple environments. The success of IRM requires an important assumption: the underlying causal mechanisms/features remain invariant across environments. When not satisfied, we show that IRM can over-constrain the predictor and to remedy this, we propose a relaxation via $\textit{partial invariance}$. In this work, we theoretically highlight the sub-optimality of IRM and then demonstrate how learning from a partition of training domains can help improve invariant models. Several experiments, conducted both in linear settings as well as with deep neural networks on tasks over both language and image data, allow us to verify our conclusions.

📄 PDF Abstract BibTeX arXiv:2301.12067

Code (1)

ibtihalferwana/pirm 공식 구현 pytorch

Tasks

Domain Generalization

Partial Optimal Transport with Applications on Positive-Unlabeled Learning

2020-02-19 · Laetitia Chapel, Mokhtar Z. Alaya, Gilles Gasso

Classical optimal transport problem seeks a transportation map that preserves the total mass betwenn two probability distributions, requiring their mass to be the same. This may be too restrictive in certain applications…

Partial Law Invariance and Risk Measures

2024-01-30 · Yi Shen, Zachary Van Oosten, Ruodu Wang

We introduce the concept of partial law invariance, generalizing the concepts of law invariance and probabilistic sophistication widely used in decision theory, as well as statistical and financial applications. This new…

Decision Making Decision Making Under Uncertainty Management

Model Metamers Reveal Invariances in Graph Neural Networks

2025-10-20 · Wei Xu, Xiaoyi Jiang, Lixiang Xu, Dechao Tang arxiv

In recent years, deep neural networks have been extensively employed in perceptual systems to learn representations endowed with invariances, aiming to emulate the invariance mechanisms observed in the human brain. Howev…

Data Augmentation: A Fourier Analysis Perspective

2026-06-23 · Behrooz Tahmasebi, Melanie Weber, Stefanie Jegelka arxiv

Data augmentation is a simple and model-agnostic approach for exploiting known invariances in learning problems. Given a group acting on the input space, one augments the training set with transformed copies of each samp…

Data Augmentation

Continuous Invariance Learning

2023-10-09 · Yong Lin, Fan Zhou, Lu Tan, Lintao Ma 외

Invariance learning methods aim to learn invariant features in the hope that they generalize under distributional shifts. Although many tasks are naturally characterized by continuous domains, current invariance learning…

Cloud Computing CPU