Hypernetworks for Zero-shot Transfer in Reinforcement Learning
In this paper, hypernetworks are trained to generate behaviors across a range of unseen task conditions, via a novel TD-based training objective and data from a set of near-optimal RL solutions for training tasks. This work relates to meta RL, contextual RL, and transfer learning, with a particular focus on zero-shot performance at test time, enabled by knowledge of the task parameters (also known as context). Our technical approach is based upon viewing each RL algorithm as a mapping from the MDP specifics to the near-optimal value function and policy and seek to approximate it with a hypernetwork that can generate near-optimal value functions and policies, given the parameters of the MDP. We show that, under certain conditions, this mapping can be considered as a supervised learning problem. We empirically evaluate the effectiveness of our method for zero-shot transfer to new reward and transition dynamics on a series of continuous control tasks from DeepMind Control Suite. Our method demonstrates significant improvements over baselines from multitask and meta RL approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)Transfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Brief Review of Hypernetworks in Deep Learning
Hypernetworks, or hypernets for short, are neural networks that generate weights for another neural network, known as the target network. They have emerged as a powerful deep learning technique that allows for greater fl…
Causal InferenceContinual LearningDeep LearningModel Compression+3Personalized Federated Hypernetworks for Privacy Preservation in Multi-Task Reinforcement Learning
Multi-Agent Reinforcement Learning currently focuses on implementations where all data and training can be centralized to one machine. But what if local agents are split across multiple tasks, and need to keep data priva…
Federated LearningMulti-agent Reinforcement LearningPersonalized Federated Learningreinforcement-learning+2Capability-Aware Shared Hypernetworks for Flexible Heterogeneous Multi-Robot Coordination
Recent advances have enabled heterogeneous multi-robot teams to learn complex and effective coordination skills. However, existing neural architectures that support heterogeneous teaming tend to force a trade-off between…
DiversityImitation LearningMeta-LearningTransfer Learning+1Hypernetwork Dismantling via Deep Reinforcement Learning
Network dismantling aims to degrade the connectivity of a network by removing an optimal set of nodes. It has been widely adopted in many real-world applications such as epidemic control and rumor containment. However, c…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Understanding and Simplifying One-Shot Architecture Search
There is growing interest in automating neural network architecture design. Existing architecture search methods can be computationally expensive, requiring thousands of different architectures to be trained from sc…
Neural Architecture Searchreinforcement-learningReinforcement LearningReinforcement Learning (RL)