GABRIL: Gaze-Based Regularization for Mitigating Causal Confusion in Imitation Learning

2025-07-25 · Amin Banayeeanzade, Fatemeh Bahrani, Yutai Zhou, Erdem Bıyık arxiv

Imitation Learning (IL) is a widely adopted approach which enables agents to learn from human expert demonstrations by framing the task as a supervised learning problem. However, IL often suffers from causal confusion, where agents misinterpret spurious correlations as causal relationships, leading to poor performance in testing environments with distribution shift. To address this issue, we introduce GAze-Based Regularization in Imitation Learning (GABRIL), a novel method that leverages the human gaze data gathered during the data collection phase to guide the representation learning in IL. GABRIL utilizes a regularization loss which encourages the model to focus on causally relevant features identified through expert gaze and consequently mitigates the effects of confounding variables. We validate our approach in Atari environments and the Bench2Drive benchmark in CARLA by collecting human gaze datasets and applying our method in both domains. Experimental results show that the improvement of GABRIL over behavior cloning is around 179% more than the same number for other baselines in the Atari and 76% in the CARLA setup. Finally, we show that our method provides extra explainability when compared to regular IL agents.

📄 PDF Abstract BibTeX arXiv:2507.19647

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning

Efficiently Guiding Imitation Learning Agents with Human Gaze

2020-02-28 · Akanksha Saran, Ruohan Zhang, Elaine Schaertl Short, Scott Niekum

Human gaze is known to be an intention-revealing signal in human demonstrations of tasks. In this work, we use gaze cues from human demonstrators to enhance the performance of agents trained via three popular imitation l…

Atari Games Imitation Learning Reinforcement Learning

Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning

2021-10-27 · NeurIPS 2021 12 · Jongjin Park, Younggyo Seo, Chang Liu, Li Zhao 외

Behavioral cloning has proven to be effective for learning sequential decision-making policies from expert demonstrations. However, behavioral cloning often suffers from the causal confusion problem where a policy relies…

Decision Making Imitation Learning Sequential Decision Making

See Through the Noise: Improving Domain Generalization in Gaze Estimation

2026-04-17 · Yanming Peng, Shijing Wang, Yaping Huang, Yi Tian arxiv

Generalizable gaze estimation methods have garnered increasing attention due to their critical importance in real-world applications and have achieved significant progress. However, they often overlook the effect of labe…

Domain Generalization Gaze Estimation

Resolving Causal Confusion in Reinforcement Learning via Robust Exploration

2021-03-09 · ICLR Workshop SSL-RL 2021 5 · Clare Lyle, Amy Zhang, Minqi Jiang, Joelle Pineau 외

A reinforcement learning agent must distinguish between spurious correlations and causal relationships in its environment in order to robustly achieve its goals. Contrary to popular belief, such cases of causal confusion…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

How Much Future Helps? A Controlled Study of Future-Privileged Supervision for Causal Egocentric Gaze Estimation

2026-07-01 · Jia Li, Wenjie Zhao, Fnu Atisri, Sanskriti Aripineni 외 arxiv

Egocentric gaze estimation is commonly studied using models that process the full video with access to future frames, while real-world applications require strictly causal, online prediction. This discrepancy raises key …

Gaze Estimation