LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization
Domain generalization (DG) methods aim to maintain good performance in an unseen target domain by using training data from multiple source domains. While success on certain occasions are observed, enhancing the baseline across most scenarios remains challenging. This work introduces a simple yet effective framework, dubbed learning from multiple experts (LFME), that aims to make the target model an expert in all source domains to improve DG. Specifically, besides learning the target model used in inference, LFME will also train multiple experts specialized in different domains, whose output probabilities provide professional guidance by simply regularizing the logit of the target model. Delving deep into the framework, we reveal that the introduced logit regularization term implicitly provides effects of enabling the target model to harness more information, and mining hard samples from the experts during training. Extensive experiments on benchmarks from different DG tasks demonstrate that LFME is consistently beneficial to the baseline and can achieve comparable performance to existing arts. Code is available at~\url{https://github.com/liangchen527/LFME}.
Code (1)
Tasks
Domain GeneralizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification
In real-world scenarios, data tends to exhibit a long-tailed distribution, which increases the difficulty of training deep networks. In this paper, we propose a novel self-paced knowledge distillation framework, termed L…
General ClassificationKnowledge DistillationLong-tail LearningSelfMem: Self-Optimizing Memory for AI Agents
While current AI agents support increasingly long context windows, tool use, and skill execution for long-horizon tasks, they still require memory systems to effectively leverage historical experience. Existing memory fr…
Learning Accurate Segmentation Purely from Self-Supervision
Accurately segmenting objects without any manual annotations remains one of the core challenges in computer vision. In this work, we introduce Selfment, a fully self-supervised framework that segments foreground objects …
Unsupervised Saliency DetectionZero-shot GeneralizationObject DetectionPerceptual evaluation of listener envelopment using spatial granular synthesis
Listener envelopment refers to the sensation of being surrounded by sound, either by multiple direct sound events or by a diffuse reverberant sound field. More recently, a specific attribute for the sensation of being co…
AttributeTarget shape dependence in a simple model of receptor-mediated endocytosis and phagocytosis
Phagocytosis and receptor-mediated endocytosis are vitally important particle uptake mechanisms in many cell types, ranging from single-cell organisms to immune cells. In both processes, engulfment by the cell depends cr…