Domain Generalization via Balancing Training Difficulty and Model Capability
Domain generalization (DG) aims to learn domain-generalizable models from one or multiple source domains that can perform well in unseen target domains. Despite its recent progress, most existing work suffers from the misalignment between the difficulty level of training samples and the capability of contemporarily trained models, leading to over-fitting or under-fitting in the trained generalization model. We design MoDify, a Momentum Difficulty framework that tackles the misalignment by balancing the seesaw between the model's capability and the samples' difficulties along the training process. MoDify consists of two novel designs that collaborate to fight against the misalignment while learning domain-generalizable models. The first is MoDify-based Data Augmentation which exploits an RGB Shuffle technique to generate difficulty-aware training samples on the fly. The second is MoDify-based Network Optimization which dynamically schedules the training samples for balanced and smooth learning with appropriate difficulty. Without bells and whistles, a simple implementation of MoDify achieves superior performance across multiple benchmarks. In addition, MoDify can complement existing methods as a plug-in, and it is generic and can work for different visual recognition tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationDomain GeneralizationSimilar Papers 제목 키워드 기반
Embracing the Dark Knowledge: Domain Generalization Using Regularized Knowledge Distillation
Though convolutional neural networks are widely used in different tasks, lack of generalization capability in the absence of sufficient and representative data is one of the challenges that hinder their practical applica…
Domain Generalizationimage-classificationImage ClassificationKnowledge DistillationCoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning
Visual instruction tuning is a key training stage of large multimodal models (LMMs). Nevertheless, the common practice of indiscriminately mixing instruction-following data from various tasks may result in suboptimal ove…
Instruction FollowingADHint: Adaptive Hints with Difficulty Priors for Reinforcement Learning
To address the limited capability expansion and low sample efficiency of Reinforcement Learning (RL), recent methods have integrated ''hints'' into post-training, which are prefix segments of complete reasoning trajector…
Reinforcement LearningGeneralizing to any diverse distribution: uniformity, gentle finetuning and rebalancing
As training datasets grow larger, we aspire to develop models that generalize well to any diverse test distribution, even if the latter deviates significantly from the training data. Various approaches like domain adapta…
Domain AdaptationDomain GeneralizationMoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning
The growing demand for larger-scale models in the development of \textbf{L}arge \textbf{L}anguage \textbf{M}odels (LLMs) poses challenges for efficient training within limited computational resources. Traditional fine-tu…
Multi-Task Learning