Hybrid Reinforcement Learning with Expert State Sequences

2019-03-11 · Xiaoxiao Guo, Shiyu Chang, Mo Yu, Gerald Tesauro, Murray Campbell

Existing imitation learning approaches often require that the complete demonstration data, including sequences of actions and states, are available. In this paper, we consider a more realistic and difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are unobserved. We propose a novel tensor-based model to infer the unobserved actions of the expert state sequences. The policy of the agent is then optimized via a hybrid objective combining reinforcement learning and imitation learning. We evaluated our hybrid approach on an illustrative domain and Atari games. The empirical results show that (1) the agents are able to leverage state expert sequences to learn faster than pure reinforcement learning baselines, (2) our tensor-based action inference model is advantageous compared to standard deep neural networks in inferring expert actions, and (3) the hybrid policy optimization objective is robust against noise in expert state sequences.

📄 PDF Abstract BibTeX arXiv:1903.04110

Code (1)

XiaoxiaoGuo/tensor4rl 공식 구현 pytorch

Tasks

Atari GamesImitation Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Faster Reinforcement Learning with Expert State Sequences

2018-01-01 · ICLR 2018 1 · Xiaoxiao Guo, Shiyu Chang, Mo Yu, Miao Liu 외

Imitation learning relies on expert demonstrations. Existing approaches often re- quire that the complete demonstration data, including sequences of actions and states are available. In this paper, we consider a realisti…

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Mimicking actions is a good strategy for beginners: Fast Reinforcement Learning with Expert Action Sequences

2018-09-27 · Tharun Medini, Anshumali Shrivastava

Imitation Learning is the task of mimicking the behavior of an expert player in a Reinforcement Learning(RL) Environment to enhance the training of a fresh agent (called novice) beginning from scratch. Most of the Reinfo…

Atari Games Imitation Learning reinforcement-learning Reinforcement Learning+1

Hybrid Inverse Reinforcement Learning

2024-02-13 · Juntao Ren, Gokul Swamy, Zhiwei Steven Wu, J. Andrew Bagnell 외

The inverse reinforcement learning approach to imitation learning is a double-edged sword. On the one hand, it can enable learning from a smaller number of expert demonstrations with more robustness to error compounding …

continuous-control Continuous Control Imitation Learning reinforcement-learning+2

Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learning

2026-03-16 · Jing Ye, Xinpei Zhao, Lu Xiang, Yaping Zhang 외 arxiv

While current emotional support dialogue systems typically rely on expert-defined scalar rewards for alignment, these signals suffer from severe information sparsity. They cannot explain why a response failed or how to a…

Reinforcement Learning

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics

2020-01-02 · Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier, Thomas Lampe 외

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)