Meta Reinforcement Learning with Distribution of Exploration Parameters Learned by Evolution Strategies

2018-12-29 · Yiming Shen, Kehan Yang, Yufeng Yuan, Simon Cheng Liu

In this paper, we propose a novel meta-learning method in a reinforcement learning setting, based on evolution strategies (ES), exploration in parameter space and deterministic policy gradients. ES methods are easy to parallelize, which is desirable for modern training architectures; however, such methods typically require a huge number of samples for effective training. We use deterministic policy gradients during adaptation and other techniques to compensate for the sample-efficiency problem while maintaining the inherent scalability of ES methods. We demonstrate that our method achieves good results compared to gradient-based meta-learning in high-dimensional control tasks in the MuJoCo simulator. In addition, because of gradient-free methods in the meta-training phase, which do not need information about gradients and policies in adaptation training, we predict and confirm our algorithm performs better in tasks that need multi-step adaptation.

📄 PDF Abstract BibTeX arXiv:1812.11314

Code (0)

등록된 구현이 없습니다.

Tasks

Meta-LearningMeta Reinforcement LearningMuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation

2019-10-30 · NeurIPS 2019 12 · Risto Vuorio, Shao-Hua Sun, Hexiang Hu, Joseph J. Lim

Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those fra…

Few-Shot Image Classification Few-Shot Learning General Classification image-classification+5

Emergence of Collective Open-Ended Exploration from Decentralized Meta-Reinforcement Learning

2023-11-01 · Richard Bornemann, Gautier Hamon, Eleni Nisioti, Clément Moulin-Frier

Recent works have proven that intricate cooperative behaviors can emerge in agents trained using meta reinforcement learning on open ended task distributions using self-play. While the results are impressive, we argue th…

Meta Reinforcement Learning reinforcement-learning

Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning

2017-11-04 · Richard Liaw, Sanjay Krishnan, Animesh Garg, Daniel Crankshaw 외

Rather than learning new control policies for each new task, it is possible, when tasks share some structure, to compose a "meta-policy" from previously learned policies. This paper reports results from experiments using…

Autonomous Driving Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Celo2: Towards Learned Optimization Free Lunch

2026-02-22 · Abhinav Moudgil, Boris Knyazev, Eugene Belilovsky arxiv

Learned optimizers are powerful alternatives to hand-designed update rules like Adam, yet they have seen limited practical adoption since they often fail to meta-generalize beyond their training distribution and incur hi…

Meta-Reinforcement Learning by Tracking Task Non-stationarity

2021-05-18 · Riccardo Poiani, Andrea Tirinzoni, Marcello Restelli

Many real-world domains are subject to a structured non-stationarity which affects the agent's goals and the environmental dynamics. Meta-reinforcement learning (RL) has been shown successful for training agents that qui…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)