paper-with-me

홈 › Papers

Dispensed Transformer Network for Unsupervised Domain Adaptation

2021-10-28 · Yunxiang Li, Jingxiong Li, Ruilong Dan, Shuai Wang, Kai Jin, Guodong Zeng, Jun Wang, Xiangji Pan, Qianni Zhang, Huiyu Zhou, Qun Jin, Li Wang, Yaqi Wang

Accurate segmentation is a crucial step in medical image analysis and applying supervised machine learning to segment the organs or lesions has been substantiated effective. However, it is costly to perform data annotation that provides ground truth labels for training the supervised algorithms, and the high variance of data that comes from different domains tends to severely degrade system performance over cross-site or cross-modality datasets. To mitigate this problem, a novel unsupervised domain adaptation (UDA) method named dispensed Transformer network (DTNet) is introduced in this paper. Our novel DTNet contains three modules. First, a dispensed residual transformer block is designed, which realizes global attention by dispensed interleaving operation and deals with the excessive computational cost and GPU memory usage of the Transformer. Second, a multi-scale consistency regularization is proposed to alleviate the loss of details in the low-resolution output for better feature alignment. Finally, a feature ranking discriminator is introduced to automatically assign different weights to domain-gap features to lessen the feature distribution distance, reducing the performance shift of two domains. The proposed method is evaluated on large fluorescein angiography (FA) retinal nonperfusion (RNP) cross-site dataset with 676 images and a wide used cross-modality dataset from the MM-WHS challenge. Extensive results demonstrate that our proposed network achieves the best performance in comparison with several state-of-the-art techniques.

📄 PDF Abstract BibTeX arXiv:2110.14944

Code (0)

등록된 구현이 없습니다.

Tasks

Domain AdaptationGPUMedical Image AnalysisUnsupervised Domain Adaptation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Position-Wise Feed-Forward Layer 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Similar Papers 제목 키워드 기반

Automatic Domain Adaptation by Transformers in In-Context Learning

2024-05-27 · Ryuichiro Hataya, Kota Matsui, Masaaki Imaizumi

Selecting or designing an appropriate domain adaptation algorithm for a given problem remains challenging. This paper presents a Transformer model that can provably approximate and opt for domain adaptation methods for a…

Domain AdaptationIn-Context LearningUnsupervised Domain Adaptation

Cross-View Cross-Modal Unsupervised Domain Adaptation for Driver Monitoring System

2025-11-15 · Aditi Bhalla, Christian Hellert, Enkelejda Kasneci arxiv

Driver distraction remains a leading cause of road traffic accidents, contributing to thousands of fatalities annually across the globe. While deep learning-based driver activity recognition methods have shown promise in…

Unsupervised Domain AdaptationContrastive LearningActivity Recognition

TransAdapter: Vision Transformer for Feature-Centric Unsupervised Domain Adaptation

2024-12-05 · A. Enes Doruk, Erhan Oztop, Hasan F. Ates

Unsupervised Domain Adaptation (UDA) aims to utilize labeled data from a source domain to solve tasks in an unlabeled target domain, often hindered by significant domain gaps. Traditional CNN-based methods struggle to fu…

Domain AdaptationUnsupervised Domain Adaptation

EUDA: An Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer

2024-07-31 · Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah

Unsupervised domain adaptation (UDA) aims to mitigate the domain shift issue, where the distribution of training (source) data differs from that of testing (target) data. Many models have been developed to tackle this pr…

Domain AdaptationUnsupervised Domain Adaptation

Unsupervised Domain Adaptation within Deep Foundation Latent Spaces

2024-02-22 · Dmitry Kangin, Plamen Angelov

The vision transformer-based foundation models, such as ViT or Dino-V2, are aimed at solving problems with little or no finetuning of features. Using a setting of prototypical networks, we analyse to what extent such fou…

Decision MakingDomain AdaptationUnsupervised Domain Adaptation