Meta-Gradient Reinforcement Learning

2018-05-24 · NeurIPS 2018 12 · Zhongwen Xu, Hado van Hasselt, David Silver

The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide the true value function. Instead, the majority of reinforcement learning algorithms estimate and/or optimise a proxy for the value function. This proxy is typically based on a sampled and bootstrapped approximation to the true value function, known as a return. The particular choice of return is one of the chief components determining the nature of the algorithm: the rate at which future rewards are discounted; when and how values should be bootstrapped; or even the nature of the rewards themselves. It is well-known that these decisions are crucial to the overall success of RL algorithms. We discuss a gradient-based meta-learning algorithm that is able to adapt the nature of the return, online, whilst interacting and learning from the environment. When applied to 57 games on the Atari 2600 environment over 200 million frames, our algorithm achieved a new state-of-the-art performance.

📄 PDF Abstract BibTeX arXiv:1805.09801

Code (1)

RobvanGastel/meta-rl-algorithms pytorch

Tasks

Meta-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Curriculum in Gradient-Based Meta-Reinforcement Learning

2020-02-19 · Bhairav Mehta, Tristan Deleu, Sharath Chandra Raparthy, Chris J. Pal 외

Gradient-based meta-learners such as Model-Agnostic Meta-Learning (MAML) have shown strong few-shot performance in supervised and reinforcement learning settings. However, specifically in the case of meta-reinforcement l…

Benchmarking Meta-Learning Meta Reinforcement Learning Out-of-Distribution Generalization+3

Debiasing Meta-Gradient Reinforcement Learning by Learning the Outer Value Function

2022-11-19 · Clément Bonnet, Laurence Midgley, Alexandre Laterre

Meta-gradient Reinforcement Learning (RL) allows agents to self-tune their hyper-parameters in an online fashion during training. In this paper, we identify a bias in the meta-gradient of current meta-gradient RL approac…

reinforcement-learning Reinforcement Learning (RL)

On the Convergence Theory of Debiased Model-Agnostic Meta-Reinforcement Learning

2020-02-12 · NeurIPS 2021 12 · Alireza Fallah, Kristian Georgiev, Aryan Mokhtari, Asuman Ozdaglar

We consider Model-Agnostic Meta-Learning (MAML) methods for Reinforcement Learning (RL) problems, where the goal is to find a policy using data from several tasks represented by Markov Decision Processes (MDPs) that can …

Meta-Learning Meta Reinforcement Learning Policy Gradient Methods reinforcement-learning+2

Interpretable Meta-Reinforcement Learning with Actor-Critic Method

2021-01-01 · Xingyuan Liang, Xu-Ying Liu

Meta-reinforcement learning (meta-RL) algorithms have successfully trained agent systems to perform well on different tasks within only few updates. However, in gradient-based meta-RL algorithms, the Q-function at adapta…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation

2025-09-30 · Yang Zhang, Huiwen Yan, Mushuang Liu arxiv

Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents …

Computational Efficiency Reinforcement Learning