Meta-Model-Based Meta-Policy Optimization
Model-based meta-reinforcement learning (RL) methods have recently been shown to be a promising approach to improving the sample efficiency of RL in multi-task settings. However, the theoretical understanding of those methods is yet to be established, and there is currently no theoretical guarantee of their performance in a real-world environment. In this paper, we analyze the performance guarantee of model-based meta-RL methods by extending the theorems proposed by Janner et al. (2019). On the basis of our theoretical results, we propose Meta-Model-Based Meta-Policy Optimization (M3PO), a model-based meta-RL method with a performance guarantee. We demonstrate that M3PO outperforms existing meta-RL methods in continuous-control benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlMeta-LearningMeta Reinforcement Learningmodelreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Model-Based Offline Meta-Reinforcement Learning with Regularization
Existing offline reinforcement learning (RL) methods face a few major challenges, particularly the distributional shift between the learned policy and the behavior policy. Offline Meta-RL is emerging as a promising appro…
Meta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Double Meta-Learning for Data Efficient Policy Optimization in Non-Stationary Environments
We are interested in learning models of non-stationary environments, which can be framed as a multi-task learning problem. Model-free reinforcement learning algorithms can achieve good asymptotic performance in multi-tas…
Meta-LearningMeta Reinforcement LearningMulti-Task Learningreinforcement-learning+2Meta-Reinforcement Learning with Universal Policy Adaptation: Provable Near-Optimality under All-task Optimum Comparator
Meta-reinforcement learning (Meta-RL) has attracted attention due to its capability to enhance reinforcement learning (RL) algorithms, in terms of data efficiency and generalizability. In this paper, we develop a bilevel…
AllBilevel OptimizationMeta Reinforcement Learningreinforcement-learning+2On First-Order Meta-Reinforcement Learning with Moreau Envelopes
Meta-Reinforcement Learning (MRL) is a promising framework for training agents that can quickly adapt to new environments and tasks. In this work, we study the MRL problem under the policy gradient formulation, where we …
Meta Reinforcement Learningreinforcement-learningReinforcement LearningSurrogate Learning in Meta-Black-Box Optimization: A Preliminary Study
Recent Meta-Black-Box Optimization (MetaBBO) approaches have shown possibility of enhancing the optimization performance through learning meta-level policies to dynamically configure low-level optimizers. However, existi…
Kolmogorov-Arnold NetworksReinforcement Learning (RL)