An Empirical Study of Invariant Risk Minimization

2020-04-10 · Yo Joong Choe, Jiyeon Ham, Kyubyong Park

Invariant risk minimization (IRM) (Arjovsky et al., 2019) is a recently proposed framework designed for learning predictors that are invariant to spurious correlations across different training environments. Yet, despite its theoretical justifications, IRM has not been extensively tested across various settings. In an attempt to gain a better understanding of the framework, we empirically investigate several research questions using IRMv1, which is the first practical algorithm proposed to approximately solve IRM. By extending the ColoredMNIST experiment in different ways, we find that IRMv1 (i) performs better as the spurious correlation varies more widely between training environments, (ii) learns an approximately invariant predictor when the underlying relationship is approximately invariant, and (iii) can be extended to an analogous setting for text classification.

📄 PDF Abstract BibTeX arXiv:2004.05007

Code (1)

kakaobrain/irm-empirical-study 공식 구현 pytorch

Tasks

text-classification Text Classification

Continual Invariant Risk Minimization

2023-10-21 · Francesco Alesiani, Shujian Yu, Mathias Niepert

Empirical risk minimization can lead to poor generalization behavior on unseen environments if the learned model does not capture invariant feature representations. Invariant risk minimization (IRM) is a recent proposal …

Continual Learning

Kernelized Heterogeneous Risk Minimization

2021-10-24 · Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li 외

The ability to generalize under distributional shifts is essential to reliable machine learning, while models optimized with empirical risk minimization usually fail on non-$i.i.d$ testing data. Recently, invariant learn…

Bayesian Invariant Risk Minimization

2022-01-01 · CVPR 2022 1 · Yong Lin, Hanze Dong, Hao Wang, Tong Zhang

Generalization under distributional shift is an open challenge for machine learning. Invariant Risk Minimization (IRM) is a promising framework to tackle this issue by extracting invariant features. However, despite …

Bayesian Inference

Robust Reconfigurable Intelligent Surfaces via Invariant Risk and Causal Representations

2021-05-04 · Sumudu Samarakoon, Jihong Park, Mehdi Bennis

In this paper, the problem of robust reconfigurable intelligent surface (RIS) system design under changes in data distributions is investigated. Using the notion of invariant risk minimization (IRM), an invariant causal …

Integrated Latent Heterogeneity and Invariance Learning in Kernel Space

2021-12-01 · NeurIPS 2021 12 · Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li 외