Discriminator Soft Actor Critic without Extrinsic Rewards

2020-01-19 · Daichi Nishio, Daiki Kuyoshi, Toi Tsuneda, Satoshi Yamane

It is difficult to be able to imitate well in unknown states from a small amount of expert data and sampling data. Supervised learning methods such as Behavioral Cloning do not require sampling data, but usually suffer from distribution shift. The methods based on reinforcement learning, such as inverse reinforcement learning and generative adversarial imitation learning (GAIL), can learn from only a few expert data. However, they often need to interact with the environment. Soft Q imitation learning addressed the problems, and it was shown that it could learn efficiently by combining Behavioral Cloning and soft Q-learning with constant rewards. In order to make this algorithm more robust to distribution shift, we propose Discriminator Soft Actor Critic (DSAC). It uses a reward function based on adversarial inverse reinforcement learning instead of constant rewards. We evaluated it on PyBullet environments with only four expert trajectories.

📄 PDF Abstract BibTeX arXiv:2001.06808

Code (1)

dnishio/DSAC 공식 구현

Tasks

Imitation LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Adam 설명 없음

Soft Actor Critic Soft Actor Critic, or SAC, is an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN

2018-12-06 · Jogendra Nath Kundu, Maharshi Gor, R. Venkatesh Babu

Human motion prediction model has applications in various fields of computer vision. Without taking into account the inherent stochasticity in the prediction of future pose dynamics, such methods often converges to a det…

Human motion prediction motion prediction Prediction

Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator

2024-01-30 · Ryoma Furuyama, Daiki Kuyoshi, Satoshi Yamane

Imitation learning is often used in addition to reinforcement learning in environments where reward design is difficult or where the reward is sparse, but it is difficult to be able to imitate well in unknown states from…

Imitation Learning MuJoCo Q-Learning reinforcement-learning+1

Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning

2018-09-09 · ICLR 2019 5 · Ilya Kostrikov, Kumar Krishna Agrawal, Debidatta Dwibedi, Sergey Levine 외

We identify two issues with the family of algorithms based on the Adversarial Imitation Learning framework. The first problem is implicit bias present in the reward functions used in these algorithms. While these biases …

Imitation Learning Reinforcement Learning

Soft Actor-Critic for Discrete Action Settings

2019-10-16 · Petros Christodoulou

Soft Actor-Critic is a state-of-the-art reinforcement learning algorithm for continuous action settings that is not applicable to discrete action settings. Many important settings involve discrete actions, however, and s…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning

2017-10-31 · Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu 외

This paper presents a new method --- adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems. Inspired by generati…

Task-Completion Dialogue Policy Learning