홈 › Papers

Meta-Reinforcement Learning With Informed Policy Regularization

2021-01-01 · Pierre-Alexandre Kamienny, Matteo Pirotta, Alessandro Lazaric, Thibault Lavril, Nicolas Usunier, Ludovic Denoyer

Meta-reinforcement learning aims at finding a policy able to generalize to new environments. When facing a new environment, this policy must explore to identify its particular characteristics and then exploit this information for collecting reward. Even though policies based on recurrent neural networks can be used in this setting by training them on multiple environments, they often fail to model this trade-off, or solve it at a very high computational cost. In this paper, we propose a new algorithm that uses privileged information in the form of a task descriptor at train time to improve the learning of recurrent policies. Our method learns an informed policy (i.e., a policy receiving as input the description of the current task) that is used to both construct task embeddings from the descriptors, and to regularize the training of the recurrent policy through parameters sharing and an auxiliary objective. This approach significantly reduces the learning sample complexity without altering the representational power of RNNs, by focusing on the relevant characteristics of the task, and by exploiting them efficiently. We evaluate our algorithm in a variety of environments that require sophisticated exploration/exploitation strategies and show that it outperforms vanilla RNNs, Thompson sampling and the task-inference approaches to meta-reinforcement learning.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Meta Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

On First-Order Meta-Reinforcement Learning with Moreau Envelopes

2023-05-20 · Mohammad Taha Toghani, Sebastian Perez-Salazar, César A. Uribe

Meta-Reinforcement Learning (MRL) is a promising framework for training agents that can quickly adapt to new environments and tasks. In this work, we study the MRL problem under the policy gradient formulation, where we …

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning

Model-Based Offline Meta-Reinforcement Learning with Regularization

2022-02-07 · ICLR 2022 4 · Sen Lin, Jialin Wan, Tengyu Xu, Yingbin Liang 외

Existing offline reinforcement learning (RL) methods face a few major challenges, particularly the distributional shift between the learned policy and the behavior policy. Offline Meta-RL is emerging as a promising appro…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization

2021-08-19 · Lu Wen, Songan Zhang, H. Eric Tseng, Baljeet Singh 외

Meta Reinforcement Learning (Meta-RL) has seen substantial advancements recently. In particular, off-policy methods were developed to improve the data efficiency of Meta-RL techniques. \textit{Probabilistic embeddings fo…

Autonomous Vehicles Decision Making Meta Reinforcement Learning MuJoCo

Integrating Physics-Informed Neural Networks for Safe Reinforcement Learning in a 1-DoF Helicopter System

2026-07-03 · Georg Schäfer, Jakob Rehrl, Stefan Huber arxiv

Deep reinforcement learning (DRL) offers powerful control for industrial cyber-physical systems (ICPSs), but its "black-box" exploration risks violating strict hardware safety limits. Typically, these constraints are man…

Reinforcement Learning

Meta Reinforcement Learning for Optimal Design of Legged Robots

2022-10-06 · Álvaro Belmonte-Baeza, Joonho Lee, Giorgio Valsecchi, Marco Hutter

The process of robot design is a complex task and the majority of design decisions are still based on human intuition or tedious manual tuning. A more informed way of facing this task is computational design methods wher…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)