Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator

2024-01-30 · Ryoma Furuyama, Daiki Kuyoshi, Satoshi Yamane

Imitation learning is often used in addition to reinforcement learning in environments where reward design is difficult or where the reward is sparse, but it is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement learning and Generative Adversarial imitation learning (GAIL), can learn from only a few expert data. However, they often need to interact with the environment. Soft Q imitation learning (SQIL) addressed the problems, and it was shown that it could learn efficiently by combining Behavioral Cloning and soft Q-learning with constant rewards. In order to make this algorithm more robust to distribution shift, we propose more efficient and robust algorithm by adding to this method a reward function based on adversarial inverse reinforcement learning that rewards the agent for performing actions in status similar to the demo. We call this algorithm Discriminator Soft Q Imitation Learning (DSQIL). We evaluated it on MuJoCo environments.

📄 PDF Abstract BibTeX arXiv:2401.16772

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation LearningMuJoCoQ-Learningreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Unpaired Image Enhancement Featuring Reinforcement-Learning-Controlled Image Editing Software

2019-12-17 · Satoshi Kosugi, Toshihiko Yamasaki

This paper tackles unpaired image enhancement, a task of learning a mapping function which transforms input images into enhanced images in the absence of input-output image pairs. Our method is based on generative advers…

Image Enhancement reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Competitive Physics Informed Networks

2022-04-23 · Qi Zeng, Yash Kothari, Spencer H. Bryngelson, Florian Schäfer

Neural networks can be trained to solve partial differential equations (PDEs) by using the PDE residual as the loss function. This strategy is called "physics-informed neural networks" (PINNs), but it currently cannot pr…

Discriminator Soft Actor Critic without Extrinsic Rewards

2020-01-19 · Daichi Nishio, Daiki Kuyoshi, Toi Tsuneda, Satoshi Yamane

It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer f…

Imitation Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization

2020-06-23 · NeurIPS 2020 12 · Paul Barde, Julien Roy, Wonseok Jeon, Joelle Pineau 외

Adversarial Imitation Learning alternates between learning a discriminator -- which tells apart expert's demonstrations from generated ones -- and a generator's policy to produce trajectories that can fool this discrimin…

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control

2021-05-21 · Pei Xu, Ioannis Karamouzas

We present a simple and intuitive approach for interactive control of physically simulated characters. Our work builds upon generative adversarial networks (GAN) and reinforcement learning, and introduces an imitation le…

Imitation Learning Motion Generation