Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch

2020-07-02 · NeurIPS 2021 12 · Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban, Adrian Weller, Volkan Cevher

We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner. Specifically, we consider the Maximum Causal Entropy (MCE) IRL learner model and provide a tight upper bound on the learner's performance degradation based on the $\ell_1$-distance between the transition dynamics of the expert and the learner. Leveraging insights from the Robust RL literature, we propose a robust MCE IRL algorithm, which is a principled approach to help with this mismatch. Finally, we empirically demonstrate the stable performance of our algorithm compared to the standard MCE IRL algorithm under transition dynamics mismatches in both finite and continuous MDP problems.

📄 PDF Abstract BibTeX arXiv:2007.01174

Code (1)

lviano/robustmce_irl 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Cross-Domain Offline Policy Adaptation via Selective Transition Correction

2026-02-05 · Mengbei Yan, Jiafei Lyu, Shengjie Sun, Zhongjian Qiao 외 arxiv

It remains a critical challenge to adapt policies across domains with mismatched dynamics in reinforcement learning (RL). In this paper, we study cross-domain offline RL, where an offline dataset from another similar sou…

Reinforcement Learning Offline RL

Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning

2026-02-24 · Zhangjie Xia, Yu Yang, Pan Xu arxiv

Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically add…

Reinforcement Learning Domain Adaptation Offline RL

Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics

2016-04-13 · Michael Herman, Tobias Gindele, Jörg Wagner, Felix Schmitt 외

Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Modular Transfer Learning with Transition Mismatch Compensation for Excessive Disturbance Rejection

2020-07-29 · Tianming Wang, Wen-jie Lu, Huan Yu, Dikai Liu

Underwater robots in shallow waters usually suffer from strong wave forces, which may frequently exceed robot's control constraints. Learning-based controllers are suitable for disturbance rejection control, but the exce…

Transfer Learning Transfer Reinforcement Learning

Inverse Transition Learning: Learning Dynamics from Demonstrations

2024-11-07 · Leo Benac, Abhishek Sharma, Sonali Parbhoo, Finale Doshi-Velez

We consider the problem of estimating the transition dynamics $T^*$ from near-optimal expert trajectories in the context of offline model-based reinforcement learning. We develop a novel constraint-based method, Inverse …

Decision Making Management Model-based Reinforcement Learning