Effect-Invariant Mechanisms for Policy Generalization

2023-06-19 · Sorawit Saengkyongam, Niklas Pfister, Predrag Klasnja, Susan Murphy, Jonas Peters

Policy learning is an important component of many real-world learning systems. A major challenge in policy learning is how to adapt efficiently to unseen environments or tasks. Recently, it has been suggested to exploit invariant conditional distributions to learn models that generalize better to unseen environments. However, assuming invariance of entire conditional distributions (which we call full invariance) may be too strong of an assumption in practice. In this paper, we introduce a relaxation of full invariance called effect-invariance (e-invariance for short) and prove that it is sufficient, under suitable assumptions, for zero-shot policy generalization. We also discuss an extension that exploits e-invariance when we have a small sample from the test environment, enabling few-shot policy generalization. Our work does not assume an underlying causal graph or that the data are generated by a structural causal model; instead, we develop testing procedures to test e-invariance directly from data. We present empirical results using simulated data and a mobile health intervention dataset to demonstrate the effectiveness of our approach.

📄 PDF Abstract BibTeX arXiv:2306.10983

Code (0)

등록된 구현이 없습니다.

Invariant Causal Routing for Governing Social Norms in Online Market Economies

2026-03-04 · Xiangning Yu, Qirui Mi, Xiao Xue, Haoxuan Li 외 arxiv

Social norms are stable behavioral patterns that emerge endogenously within economic systems through repeated interactions among agents. In online market economies, such norms -- like fair exposure, sustained participati…

Deep Sets for Generalization in RL

2020-03-20 · Tristan Karch, Cédric Colas, Laetitia Teodorescu, Clément Moulin-Frier 외

This paper investigates the idea of encoding object-centered representations in the design of the reward function and policy architectures of a language-guided reinforcement learning agent. This is done using a combinati…

Navigate Object reinforcement-learning Reinforcement Learning+2

Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning

2020-06-01 · Anoopkumar Sonar, Vincent Pacelli, Anirudha Majumdar

A fundamental challenge in reinforcement learning is to learn policies that generalize beyond the operating domains experienced during training. In this paper, we approach this challenge through the following invariance …

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms

2021-06-02 · Yunqi Wang, Furui Liu, Zhitang Chen, Qing Lian 외

Domain generalization aims to learn knowledge invariant across different distributions while semantically meaningful for downstream tasks from multiple source domains, to improve the model's generalization ability on uns…

Domain Generalization

Target Conditioned Representation Independence (TCRI); From Domain-Invariant to Domain-General Representations

2022-12-21 · Olawale Salaudeen, Oluwasanmi Koyejo

We propose a Target Conditioned Representation Independence (TCRI) objective for domain generalization. TCRI addresses the limitations of existing domain generalization methods due to incomplete constraints. Specifically…