Reward function shape exploration in adversarial imitation learning: an empirical study
For adversarial imitation learning algorithms (AILs), no true rewards are obtained from the environment for learning the strategy. However, the pseudo rewards based on the output of the discriminator are still required. Given the implicit reward bias problem in AILs, we design several representative reward function shapes and compare their performances by large-scale experiments. To ensure our results' reliability, we conduct the experiments on a series of Mujoco and Box2D continuous control tasks based on four different AILs. Besides, we also compare the performance of various reward function shapes using varying numbers of expert trajectories. The empirical results reveal that the positive logarithmic reward function works well in typical continuous control tasks. In contrast, the so-called unbiased reward function is limited to specific kinds of tasks. Furthermore, several designed reward functions perform excellently in these environments as well.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlImitation LearningMuJoCoSimilar Papers 제목 키워드 기반
Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
The generative adversarial imitation learning (GAIL) has provided an adversarial learning framework for imitating expert policy from demonstrations in high-dimensional continuous tasks. However, almost all GAIL and its e…
continuous-controlContinuous ControlImitation LearningMuJoCoGenerative Adversarial Self-Imitation Learning
This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversa…
Imitation LearningMuJoCoreinforcement-learningReinforcement Learning+1ARC -- Actor Residual Critic for Adversarial Imitation Learning
Adversarial Imitation Learning (AIL) is a class of popular state-of-the-art Imitation Learning algorithms commonly used in robotics. In AIL, an artificial adversary's misclassification is used as a reward signal that is …
ARCcontinuous-controlContinuous ControlImitation Learning+1Guiding Pretraining in Reinforcement Learning with Large Language Models
Reinforcement learning algorithms typically struggle in the absence of a dense, well-shaped reward function. Intrinsically motivated exploration methods address this limitation by rewarding agents for visiting novel stat…
Common Sense ReasoningLanguage ModelingLanguage Modellingreinforcement-learning+2Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learning
We study the problem of reinforcement learning in infinite-horizon discounted linear Markov decision processes (MDPs), and propose the first computationally efficient algorithm achieving near-optimal regret guarantees in…
Imitation Learning