Directed-MAML: Meta Reinforcement Learning Algorithm with Task-directed Approximation

2025-09-30 · Yang Zhang, Huiwen Yan, Mushuang Liu arxiv

Model-Agnostic Meta-Learning (MAML) is a versatile meta-learning framework applicable to both supervised learning and reinforcement learning (RL). However, applying MAML to meta-reinforcement learning (meta-RL) presents notable challenges. First, MAML relies on second-order gradient computations, leading to significant computational and memory overhead. Second, the nested structure of optimization increases the problem's complexity, making convergence to a global optimum more challenging. To overcome these limitations, we propose Directed-MAML, a novel task-directed meta-RL algorithm. Before the second-order gradient step, Directed-MAML applies an additional first-order task-directed approximation to estimate the effect of second-order gradients, thereby accelerating convergence to the optimum and reducing computational cost. Experimental results demonstrate that Directed-MAML surpasses MAML-based baselines in computational efficiency and convergence speed in the scenarios of CartPole-v1, LunarLander-v2 and two-vehicle intersection crossing. Furthermore, we show that task-directed approximation can be effectively integrated into other meta-learning algorithms, such as First-Order Model-Agnostic Meta-Learning (FOMAML) and Meta Stochastic Gradient Descent(Meta-SGD), yielding improved computational efficiency and convergence speed.

📄 PDF Abstract BibTeX arXiv:2510.00212

Code (0)

등록된 구현이 없습니다.

Tasks

Computational Efficiency Reinforcement Learning

Some Considerations on Learning to Explore via Meta-Reinforcement Learning

2018-03-03 · ICLR 2018 1 · Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen 외

We consider the problem of exploration in meta reinforcement learning. Two new meta reinforcement learning algorithms are suggested: E-MAML and E-$\text{RL}^2$. Results are presented on a novel environment we call `Krazy…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Constrained Meta Agnostic Reinforcement Learning

2024-06-20 · Karam Daaboul, Florian Kuhm, Tim Joseph, J. Marius Zoellner

Meta-Reinforcement Learning (Meta-RL) aims to acquire meta-knowledge for quick adaptation to diverse tasks. However, applying these policies in real-world environments presents a significant challenge in balancing rapid …

Meta-Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

2019-05-17 · Harkirat Singh Behl, Atılım Güneş Baydin, Philip H. S. Torr

Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on fe…

Few-Shot Learning General Classification Meta-Learning model+1

The Importance of Sampling inMeta-Reinforcement Learning

2018-12-01 · NeurIPS 2018 12 · Bradly Stadie, Ge Yang, Rein Houthooft, Peter Chen 외

We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learni…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

B-SMALL: A Bayesian Neural Network approach to Sparse Model-Agnostic Meta-Learning

2021-01-01 · Anish Madan, Ranjitha Prasad

There is a growing interest in the learning-to-learn paradigm, also known as meta-learning, where models infer on new tasks using a few training examples. Recently, meta-learning based methods have been widely used in fe…

Domain Adaptation Meta-Learning regression