paper-with-me

홈 › Papers

Hybrid Inverse Reinforcement Learning

2024-02-13 · Juntao Ren, Gokul Swamy, Zhiwei Steven Wu, J. Andrew Bagnell, Sanjiban Choudhury

The inverse reinforcement learning approach to imitation learning is a double-edged sword. On the one hand, it can enable learning from a smaller number of expert demonstrations with more robustness to error compounding than behavioral cloning approaches. On the other hand, it requires that the learner repeatedly solve a computationally expensive reinforcement learning (RL) problem. Often, much of this computation is wasted searching over policies very dissimilar to the expert's. In this work, we propose using hybrid RL -- training on a mixture of online and expert data -- to curtail unnecessary exploration. Intuitively, the expert data focuses the learner on good states during training, which reduces the amount of exploration required to compute a strong policy. Notably, such an approach doesn't need the ability to reset the learner to arbitrary states in the environment, a requirement of prior work in efficient inverse RL. More formally, we derive a reduction from inverse RL to expert-competitive RL (rather than globally optimal RL) that allows us to dramatically reduce interaction during the inner policy search loop while maintaining the benefits of the IRL approach. This allows us to derive both model-free and model-based hybrid inverse RL algorithms with strong policy performance guarantees. Empirically, we find that our approaches are significantly more sample efficient than standard inverse RL and several other baselines on a suite of continuous control tasks.

📄 PDF Abstract BibTeX arXiv:2402.08848

Code (1)

jren03/garage 공식 구현 pytorch

Tasks

continuous-controlContinuous ControlImitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)

Similar Papers 제목 키워드 기반

Hybrid Policies Using Inverse Rewards for Reinforcement Learning

2018-09-27 · Yao Shi, Tian Xia, Guanjun Zhao, Xin Gao

This paper puts forward a broad-spectrum improvement for reinforcement learning algorithms, which combines the policies using original rewards and inverse (negative) rewards. The policies using inverse rewards are compet…

OpenAI GymQ-Learningreinforcement-learningReinforcement Learning+1

Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidance

2025-11-26 · Bram Silue, Santiago Amaya-Corredor, Patrick Mannion, Lander Willem 외 arxiv

Adversarial Inverse Reinforcement Learning (AIRL) has shown promise in addressing the sparse reward problem in reinforcement learning (RL) by inferring dense reward functions from expert demonstrations. However, its perf…

Reinforcement Learning

Hybrid Supervised and Reinforcement Learning for the Design and Optimization of Nanophotonic Structures

2022-09-08 · Christopher Yeung, Benjamin Pham, Zihan Zhang, Katherine T. Fountaine 외

From higher computational efficiency to enabling the discovery of novel and complex structures, deep learning has emerged as a powerful framework for the design and optimization of nanophotonic circuits and components. H…

Computational Efficiencyreinforcement-learningReinforcement LearningReinforcement Learning (RL)

Multiagent-based Participatory Urban Simulation through Inverse Reinforcement Learning

2017-12-21 · Soma Suzuki

The multiagent-based participatory simulation features prominently in urban planning as the acquired model is considered as the hybrid system of the domain and the local knowledge. However, the key problem of generating …

reinforcement-learningReinforcement LearningReinforcement Learning (RL)

Inverse Manipulation through Symbolic Planning and Residual Operator Learning

2026-06-03 · Yigit Yildirim, Giuseppe Rauso, Riccardo Caccavale, Alberto Finzi arxiv

Inverting a robotic task requires more than reversing symbolic state transitions or rewinding motor trajectories. In robot manipulation tasks, symbolic inverse plans often fail to fully restore the effects of forward exe…

Reinforcement LearningRobot Manipulation