Adapting Self-Supervised Representations to Multi-Domain Setups
Current state-of-the-art self-supervised approaches, are effective when trained on individual domains but show limited generalization on unseen domains. We observe that these models poorly generalize even when trained on a mixture of domains, making them unsuitable to be deployed under diverse real-world setups. We therefore propose a general-purpose, lightweight Domain Disentanglement Module (DDM) that can be plugged into any self-supervised encoder to effectively perform representation learning on multiple, diverse domains with or without shared classes. During pre-training according to a self-supervised loss, DDM enforces a disentanglement in the representation space by splitting it into a domain-variant and a domain-invariant portion. When domain labels are not available, DDM uses a robust clustering approach to discover pseudo-domains. We show that pre-training with DDM can show up to 3.5% improvement in linear probing accuracy on state-of-the-art self-supervised models including SimCLR, MoCo, BYOL, DINO, SimSiam and Barlow Twins on multi-domain benchmarks including PACS, DomainNet and WILDS. Models trained with DDM show significantly improved generalization (7.4%) to unseen domains compared to baselines. Therefore, DDM can efficiently adapt self-supervised encoders to provide high-quality, generalizable representations for diverse multi-domain data.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency
Visual domain adaptation (DA) seeks to transfer trained models to unseen, unlabeled domains across distribution shift, but approaches typically focus on adapting convolutional neural network architectures initialized wit…
Domain AdaptationObject RecognitionSelf-Supervised LearningLAVA: Label-efficient Visual Learning and Adaptation
We present LAVA, a simple yet effective method for multi-domain visual transfer learning with limited data. LAVA builds on a few recent innovations to enable adapting to partially labelled datasets with class and domain …
Few-Shot LearningTransfer LearningAudio Barlow Twins: Self-Supervised Audio Representation Learning
The Barlow Twins self-supervised learning objective requires neither negative samples or asymmetric learning updates, achieving results on a par with the current state-of-the-art within Computer Vision. As such, we prese…
Environmental Sound ClassificationEvent DetectionRepresentation LearningSelf-Supervised LearningSelfReplay: Adapting Self-Supervised Sensory Models via Adaptive Meta-Task Replay
Self-supervised learning has emerged as a method for utilizing massive unlabeled data for pre-training models, providing an effective feature extractor for various mobile sensing applications. However, when deployed to e…
Computational EfficiencyDomain AdaptationMeta-LearningSelf-Supervised LearningLanguage-based Action Concept Spaces Improve Video Self-Supervised Learning
Recent contrastive language image pre-training has led to learning highly transferable and robust image representations. However, adapting these models to video domains with minimal supervision remains an open problem. W…
Action RecognitionConcept AlignmentSelf-Supervised Action Recognition LinearSelf-Supervised Learning