Unsupervised Meta-Learning for Reinforcement Learning
Meta-learning algorithms use past experience to learn to quickly solve new tasks. In the context of reinforcement learning, meta-learning algorithms acquire reinforcement learning procedures to solve new problems more efficiently by utilizing experience from prior tasks. The performance of meta-learning algorithms depends on the tasks available for meta-training: in the same way that supervised learning generalizes best to test points drawn from the same distribution as the training points, meta-learning methods generalize best to tasks from the same distribution as the meta-training tasks. In effect, meta-reinforcement learning offloads the design burden from algorithm design to task design. If we can automate the process of task design as well, we can devise a meta-learning algorithm that is truly automated. In this work, we take a step in this direction, proposing a family of unsupervised meta-learning algorithms for reinforcement learning. We motivate and describe a general recipe for unsupervised meta-reinforcement learning, and present an instantiation of this approach. Our conceptual and theoretical contributions consist of formulating the unsupervised meta-reinforcement learning problem and describing how task proposals based on mutual information can be used to train optimal meta-learners. Our experimental results indicate that unsupervised meta-reinforcement learning effectively acquires accelerated reinforcement learning procedures without the need for manual task design and these procedures exceed the performance of learning from scratch.
Code (0)
등록된 구현이 없습니다.
Tasks
Meta-LearningMeta Reinforcement LearningMulti-Task Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Unsupervised Curricula for Visual Meta-Reinforcement Learning
In principle, meta-reinforcement learning algorithms leverage experience across many tasks to learn fast reinforcement learning (RL) strategies that transfer to similar tasks. However, current meta-RL approaches rely on …
ClusteringMeta-LearningMeta Reinforcement Learningreinforcement-learning+2Unsupervised Meta-Testing with Conditional Neural Processes for Hybrid Meta-Reinforcement Learning
We introduce Unsupervised Meta-Testing with Conditional Neural Processes (UMCNP), a novel hybrid few-shot meta-reinforcement learning (meta-RL) method that uniquely combines, yet distinctly separates, parameterized polic…
continuous-controlContinuous ControlMeta Reinforcement LearningUnsupervised Meta-Learning via Dynamic Head and Heterogeneous Task Construction for Few-Shot Classification
Meta-learning has been widely used in recent years in areas such as few-shot learning and reinforcement learning. However, the questions of why and when it is better than other algorithms in few-shot classification remai…
Few-Shot LearningMeta-LearningDisentangled Predictive Representation for Meta-Reinforcement Learning
A major challenge in reinforcement learning is the design of agents that are able to generalize across tasks that share common dynamics. A viable solution is meta-reinforcement learning, which identifies common struc…
Meta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Offline Meta-Reinforcement Learning with Online Self-Supervision
Meta-reinforcement learning (RL) methods can meta-train policies that adapt to new tasks with orders of magnitude less data than standard RL, but meta-training itself is costly and time-consuming. If we can meta-train on…
Meta Reinforcement LearningOffline RLreinforcement-learningReinforcement Learning+1