A Rate-Distortion Approach to Domain Generalization
Domain generalization deals with the difference in the distribution between the training and testing datasets, i.e., the domain shift problem, by extracting domain-invariant features. In this paper, we propose an information-theoretic approach for domain generalization. We first establish the domain transformation model, mapping a domain-free latent image into a domain. Then, we cast the domain generalization as a rate-distortion problem, and use the information bottleneck penalty to measure how well the domain-free latent image is reconstructed from a compressed representation of a domain-specific image compared to its direct prediction from the domain-specific image itself. We prove that the information bottleneck penalty guarantees that domain-invariant features can be learned. Lastly, we draw links of our proposed method with self-supervised contrastive learning without negative data pairs. Our empirical study on two different tasks verifies the improvement over recent baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningDomain GeneralizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bridging the Synthetic-to-Authentic Gap: Distortion-Guided Unsupervised Domain Adaptation for Blind Image Quality Assessment
The annotation of blind image quality assessment (BIQA) is labor-intensive and time-consuming, especially for authentic images. Training on synthetic data is expected to be beneficial, but synthetically trained models of…
Blind Image Quality AssessmentDomain AdaptationImage Quality AssessmentUnsupervised Domain AdaptationLook at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentation
Endeavors have been recently made to transfer knowledge from the labeled pinhole image domain to the unlabeled panoramic image domain via Unsupervised Domain Adaptation (UDA). The aim is to tackle the domain gaps caused …
Domain AdaptationERPSemantic SegmentationUnsupervised Domain AdaptationPEER pressure: Model-to-Model Regularization for Single Source Domain Generalization
Data augmentation is a popular tool for single source domain generalization, which expands the source domain by generating simulated ones, improving generalization on unseen target domains. In this work, we show that the…
Data AugmentationDomain GeneralizationmodelModel Selection+1Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability
Fine-tuning pre-trained neural network models has become a widely adopted approach across various domains. However, it can lead to the distortion of pre-trained feature extractors that already possess strong generalizati…
Signs of Struggle: Spotting Cognitive Distortions across Language and Register
Rising mental health issues among youth have increased interest in automated approaches for detecting early signs of psychological distress in digital text. One key focus is the identification of cognitive distortions, i…
Domain Adaptation