Cross-Domain Policy Adaptation by Capturing Representation Mismatch
It is vital to learn effective policies that can be transferred to different domains with dynamics discrepancies in reinforcement learning (RL). In this paper, we consider dynamics adaptation settings where there exists dynamics mismatch between the source domain and the target domain, and one can get access to sufficient source domain data, while can only have limited interactions with the target domain. Existing methods address this problem by learning domain classifiers, performing data filtering from a value discrepancy perspective, etc. Instead, we tackle this challenge from a decoupled representation learning perspective. We perform representation learning only in the target domain and measure the representation deviations on the transitions from the source domain, which we show can be a signal of dynamics mismatch. We also show that representation deviation upper bounds performance difference of a given policy in the source domain and target domain, which motivates us to adopt representation deviation as a reward penalty. The produced representations are not involved in either policy or value function, but only serve as a reward penalizer. We conduct extensive experiments on environments with kinematic and morphology mismatch, and the results show that our method exhibits strong performance on many tasks. Our code is publicly available at https://github.com/dmksjfl/PAR.
Code (1)
Tasks
Reinforcement Learning (RL)Representation LearningSimilar Papers 제목 키워드 기반
In-Context Policy Adaptation via Cross-Domain Skill Diffusion
In this work, we present an in-context policy adaptation (ICPAD) framework designed for long-horizon multi-task environments, exploring diffusion-based skill learning techniques in cross-domain settings. The framework en…
Reinforcement LearningAutonomous DrivingDomain Adaptation for Reinforcement Learning on the Atari
Deep reinforcement learning agents have recently been successful across a variety of discrete and continuous control tasks; however, they can be slow to train and require a large number of interactions with the environme…
continuous-controlContinuous ControlDeep Reinforcement LearningDomain Adaptation+4Sliced Wasserstein Discrepancy in Disentangling Representation and Adaptation Networks for Unsupervised Domain Adaptation
This paper introduces DRANet-SWD, an extension of existing work that disentangles content and style representations of images for unsupervised domain adaptation (UDA). The approach builds upon DRANet by incorporating the…
Domain AdaptationUnsupervised Domain AdaptationDomain Adaptation In Reinforcement Learning Via Latent Unified State Representation
Despite the recent success of deep reinforcement learning (RL), domain adaptation remains an open problem. Although the generalization ability of RL agents is critical for the real-world applicability of Deep RL, zero-sh…
Autonomous DrivingDeep Reinforcement LearningDomain AdaptationImage-to-Image Translation+5AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning
One practical challenge in reinforcement learning (RL) is how to make quick adaptations when faced with new environments. In this paper, we propose a principled framework for adaptive RL, called \textit{AdaRL}, that adap…
Atari Gamesreinforcement-learningReinforcement Learning (RL)Transfer Reinforcement Learning