A Unifying Framework for Causal Imitation Learning with Hidden Confounders
We propose a general and unifying framework for causal Imitation Learning (IL) with hidden confounders that subsumes several existing confounded IL settings from the literature. Our framework accounts for two types of hidden confounders: (a) those observed by the expert, which thus influence the expert's policy, and (b) confounding noise hidden to both the expert and the IL algorithm. For additional flexibility, we also introduce a confounding noise horizon and time-varying expert-observable hidden variables. We show that causal IL in our framework can be reduced to a set of Conditional Moment Restrictions (CMRs) by leveraging trajectory histories as instruments to learn a history-dependent policy. We propose DML-IL, a novel algorithm that uses instrumental variable regression to solve these CMRs and learn a policy. We provide a bound on the imitation gap for DML-IL, which recovers prior results as special cases. Empirical evaluation on a toy environment with continues state-action spaces and multiple Mujoco tasks demonstrate that DML-IL outperforms state-of-the-art causal IL algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningMuJoCoMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Causal Mediation Analysis with Hidden Confounders
An important problem in causal inference is to break down the total effect of a treatment on an outcome into different causal pathways and to quantify the causal effect in each pathway. For instance, in causal fairness, …
Causal InferenceFairnessMitigating Hidden Confounding Effects for Causal Recommendation
Recommender systems suffer from confounding biases when there exist confounders affecting both item features and user feedback (e.g., like or not). Existing causal recommendation methods typically assume confounders are …
Multi-Task LearningRecommendation SystemsGraph Disentangle Causal Model: Enhancing Causal Inference in Networked Observational Data
Estimating individual treatment effects (ITE) from observational data is a critical task across various domains. However, many existing works on ITE estimation overlook the influence of hidden confounders, which remain u…
Causal InferencecounterfactualLearning Individual Causal Effects from Networked Observational Data
Convenient access to observational data enables us to learn causal effects without randomized experiments. This research direction draws increasing attention in research areas such as economics, healthcare, and education…
Causal InferenceCausal GNNs: A GNN-Driven Instrumental Variable Approach for Causal Inference in Networks
As network data applications continue to expand, causal inference within networks has garnered increasing attention. However, hidden confounders complicate the estimation of causal effects. Most methods rely on the stron…
Causal Inference