Cross-modal Domain Adaptation for Cost-Efficient Visual Reinforcement Learning
In visual-input sim-to-real scenarios, to overcome the reality gap between images rendered in simulators and those from the real world, domain adaptation, i.e., learning an aligned representation space between simulators and the real world, then training and deploying policies in the aligned representation, is a promising direction. Previous methods focus on same-modal domain adaptation. However, those methods require building and running simulators that render high-quality images, which can be difficult and costly. In this paper, we consider a more cost-efficient setting of visual-input sim-to-real where only low-dimensional states are simulated. We first point out that the objective of learning mapping functions in previous methods that align the representation spaces is ill-posed, prone to yield an incorrect mapping. When the mapping crosses modalities, previous methods are easier to fail. Our algorithm, Cross-mOdal Domain Adaptation with Sequential structure (CODAS), mitigates the ill-posedness by utilizing the sequential nature of the data sampling process in RL tasks. Experiments on MuJoCo and Hand Manipulation Suite tasks show that the agents deployed with our method achieve similar performance as it has in the source domain, while those deployed with previous methods designed for same-modal domain adaptation suffer a larger performance gap.
Code (1)
Tasks
Domain AdaptationMuJoCoreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation
Large vision-language models (VLMs) like CLIP have demonstrated good zero-shot learning performance in the unsupervised domain adaptation task. Yet, most transfer approaches for VLMs focus on either the language or visua…
Domain AdaptationUnsupervised Domain AdaptationZero-Shot LearningCMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning
Multi-domain task-incremental learning requires a model to sequentially acquire knowledge across visually diverse domains without forgetting prior tasks, and without access to task identity at inference. Parameter-effici…
Incremental LearningOpen-Ended Visual Question Answering by Multi-Modal Domain Adaptation
We study the problem of visual question answering (VQA) in images by exploiting supervised domain adaptation, where there is a large amount of labeled data in the source domain but only limited labeled data in the target…
Domain AdaptationQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Training-Free Test-Time Adaptation with Brownian Distance Covariance in Vision-Language Models
Vision-language models suffer performance degradation under domain shift, limiting real-world applicability. Existing test-time adaptation methods are computationally intensive, rely on back-propagation, and often focus …
Test-time AdaptationStretching Domain Adaptation: How far is too far?
While deep learning has led to significant advances in visual recognition over the past few years, such advances often require a lot of annotated data. Unsupervised domain adaptation has emerged as an alternative approac…
Domain AdaptationUnsupervised Domain Adaptation