Delayed Reinforcement Learning by Imitation

2022-05-11 · Pierre Liotet, Davide Maran, Lorenzo Bisi, Marcello Restelli

When the agent's observations or interactions are delayed, classic reinforcement learning tools usually fail. In this paper, we propose a simple yet new and efficient solution to this problem. We assume that, in the undelayed environment, an efficient policy is known or can be easily learned, but the task may suffer from delays in practice and we thus want to take them into account. We present a novel algorithm, Delayed Imitation with Dataset Aggregation (DIDA), which builds upon imitation learning methods to learn how to act in a delayed environment from undelayed demonstrations. We provide a theoretical analysis of the approach that will guide the practical design of DIDA. These results are also of general interest in the delayed reinforcement learning literature by providing bounds on the performance between delayed and undelayed tasks, under smoothness conditions. We show empirically that DIDA obtains high performances with a remarkable sample efficiency on a variety of tasks, including robotic locomotion, classic control, and trading.

📄 PDF Abstract BibTeX arXiv:2205.05569

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Inverse Delayed Reinforcement Learning

2024-12-04 · Simon Sinong Zhan, Qingyuan Wu, Zhian Ruan, Frank Yang 외

Inverse Reinforcement Learning (IRL) has demonstrated effectiveness in a variety of imitation tasks. In this paper, we introduce an IRL framework designed to extract rewarding features from expert trajectories affected b…

MuJoCo reinforcement-learning Reinforcement Learning

Generative Adversarial Self-Imitation Learning

2018-12-03 · ICLR 2019 5 · Yijie Guo, Junhyuk Oh, Satinder Singh, Honglak Lee

This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversa…

Imitation Learning MuJoCo reinforcement-learning Reinforcement Learning+1

Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards

2020-10-14 · Zhixin Chen, Mengxiang Lin

The application of reinforcement learning (RL) in robotic control is still limited in the environments with sparse and delayed rewards. In this paper, we propose a practical self-imitation learning method named Self-Imit…

Imitation Learning MuJoCo Reinforcement Learning (RL)

Robust Remote Reinforcement Learning over Unreliable Communication Channels using Homomorphic State Encoding

2025-08-11 · Pietro Talli, Federico Mason, Federico Chiariotti, Andrea Zanella arxiv

Traditional Reinforcement Learning (RL) frameworks generally assume that the agent perceives the state of the underlying Markov process instantaneously and then takes actions accordingly. If the agent cannot directly obs…

Reinforcement Learning

Delay-Empowered Causal Hierarchical Reinforcement Learning

2026-05-12 · Chenran Zhao, Dianxi Shi, Haotian Wang, Mengzhu Wang 외 arxiv

Many real-world tasks involve delayed effects, where the outcomes of actions emerge after varying time lags. Existing delay-aware reinforcement learning methods often rely on state augmentation, prior knowledge of delay …

Hierarchical Reinforcement Learning