Pre-Training Transformers for Domain Adaptation

2021-12-18 · Burhan Ul Tayyab, Nicholas Chua

The Visual Domain Adaptation Challenge 2021 called for unsupervised domain adaptation methods that could improve the performance of models by transferring the knowledge obtained from source datasets to out-of-distribution target datasets. In this paper, we utilize BeiT [1] and demonstrate its capability of capturing key attributes from source datasets and apply it to target datasets in a semi-supervised manner. Our method was able to outperform current state-of-the-art (SoTA) techniques and was able to achieve 1st place on the ViSDA Domain Adaptation Challenge with ACC of 56.29% and AUROC of 69.79%.

📄 PDF Abstract BibTeX arXiv:2112.09965

Code (0)

등록된 구현이 없습니다.

Tasks

Domain Adaptation Unsupervised Domain Adaptation

Vision transformers in domain adaptation and domain generalization: a study of robustness

2024-04-05 · Shadi Alijani, Jamil Fayyad, Homayoun Najjaran

Deep learning models are often evaluated in scenarios where the data distribution is different from those used in the training and validation phases. The discrepancy presents a challenge for accurately predicting the per…

Data Augmentation Domain Adaptation Domain Generalization Meta-Learning

Feature-Weighted MMD-CORAL for Domain Adaptation in Power Transformer Fault Diagnosis

2025-05-20 · Hootan Mahmoodiyan, Maryam Ahang, Mostafa Abbasi, Homayoun Najjaran

Ensuring the reliable operation of power transformers is critical to grid stability. Dissolved Gas Analysis (DGA) is widely used for fault diagnosis, but traditional methods rely on heuristic rules, which may lead to inc…

Diagnostic Domain Adaptation Fault Diagnosis

Automatic Domain Adaptation by Transformers in In-Context Learning

2024-05-27 · Ryuichiro Hataya, Kota Matsui, Masaaki Imaizumi

Selecting or designing an appropriate domain adaptation algorithm for a given problem remains challenging. This paper presents a Transformer model that can provably approximate and opt for domain adaptation methods for a…

Domain Adaptation In-Context Learning Unsupervised Domain Adaptation

Domain-Specificity Inducing Transformers for Source-Free Domain Adaptation

2023-08-27 · ICCV 2023 1 · Sunandini Sanyal, Ashish Ramayee Asokan, Suvaansh Bhambri, Akshay Kulkarni 외

Conventional Domain Adaptation (DA) methods aim to learn domain-invariant feature representations to improve the target adaptation performance. However, we motivate that domain-specificity is equally important since in-d…

Disentanglement Domain Adaptation Source-Free Domain Adaptation Specificity

Improving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images

2023-11-21 · Gauransh Sawhney, Daksh Dave, Adeel Ahmed, Jiechao Gao 외

Unsupervised Domain Adaptation (UDA) methods facilitate knowledge transfer from a labeled source domain to an unlabeled target domain, navigating the obstacle of domain shift. While Convolutional Neural Networks (CNNs) a…

Domain Adaptation Domain Generalization Transfer Learning Unsupervised Domain Adaptation