VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning
Trading off exploration and exploitation in an unknown environment is key to maximising expected return during learning. A Bayes-optimal policy, which does so optimally, conditions its actions not only on the environment state but on the agent's uncertainty about the environment. Computing a Bayes-optimal policy is however intractable for all but the smallest tasks. In this paper, we introduce variational Bayes-Adaptive Deep RL (variBAD), a way to meta-learn to perform approximate inference in an unknown environment, and incorporate task uncertainty directly during action selection. In a grid-world domain, we illustrate how variBAD performs structured online exploration as a function of task uncertainty. We further evaluate variBAD on MuJoCo domains widely used in meta-RL and show that it achieves higher online return than existing methods.
Code (3)
Tasks
Meta-LearningMuJoCoSimilar Papers 제목 키워드 기반
Offline Meta Learning of Exploration
Consider the following instance of the Offline Meta Reinforcement Learning (OMRL) problem: given the complete training logs of $N$ conventional RL agents, trained on $N$ different tasks, design a meta-agent that can quic…
Meta-LearningMeta Reinforcement LearningOffline Meta Reinforcement Learning -- Identifiability Challenges and Effective Data Collection Strategies
Consider the following instance of the Offline Meta Reinforcement Learning (OMRL) problem: given the complete training logs of $N$ conventional RL agents, trained on $N$ different tasks, design a meta-agent that can quic…
Meta Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Meta Reinforcement Learning with Finite Training Tasks -- a Density Estimation Approach
In meta reinforcement learning (meta RL), an agent learns from a set of training tasks how to quickly solve a new task, drawn from the same task distribution. The optimal meta RL policy, a.k.a. the Bayes-optimal behavior…
Density EstimationDimensionality ReductionMeta Reinforcement Learningreinforcement-learning+1Bayesian Matrix Completion via Adaptive Relaxed Spectral Regularization
Bayesian matrix completion has been studied based on a low-rank matrix factorization formulation with promising results. However, little work has been done on Bayesian matrix completion based on the more direct spectral …
Bayesian InferenceCollaborative FilteringMatrix CompletionAmortized Bayesian Meta-Learning
Meta-learning, or learning-to-learn, has proven to be a successful strategy in attacking problems in supervised learning and reinforcement learning that involve small amounts of data. State-of-the-art solutions involve l…
Few-Shot Image ClassificationFew-Shot LearningMeta-LearningReinforcement Learning+1