Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Deep neural networks (DNNs) excel at learning representations when trained on large-scale datasets. Pre-trained DNNs also show strong transferability when fine-tuned to other labeled datasets. However, such transferability becomes weak when the target dataset is fully unlabeled as in Unsupervised Domain Adaptation (UDA). We envision that the loss of transferability may stem from the intrinsic limitation of the architecture design of DNNs. In this paper, we delve into the components of DNN architectures and propose Transferable Normalization (TransNorm) in place of existing normalization techniques. TransNorm is an end-to-end trainable layer to make DNNs more transferable across domains. As a general method, TransNorm can be easily applied to various deep neural networks and domain adaption methods, without introducing any extra hyper-parameters or learnable parameters. Empirical results justify that TransNorm not only improves classification accuracies but also accelerates convergence for mainstream DNN-based domain adaptation methods.
Code (1)
Tasks
Domain AdaptationUnsupervised Domain AdaptationSimilar Papers 제목 키워드 기반
StyLess: Boosting the Transferability of Adversarial Examples
Adversarial attacks can mislead deep neural networks (DNNs) by adding imperceptible perturbations to benign examples. The attack transferability enables adversarial examples to attack black-box DNNs with unknown architec…
Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration
Previous works have extensively studied the transferability of adversarial samples in untargeted black-box scenarios. However, it still remains challenging to craft targeted adversarial examples with higher transferabili…
Adversarial AttackLearning to Learn Transferable Generative Attack for Person Re-Identification
Deep learning-based person re-identification (re-id) models are widely employed in surveillance systems and inevitably inherit the vulnerability of deep networks to adversarial attacks. Existing attacks merely consider c…
Meta-LearningPerson Re-IdentificationQuantifying and Improving Transferability in Domain Generalization
Out-of-distribution generalization is one of the key challenges when transferring a model from the lab to the real world. Existing efforts mostly focus on building invariant features among source and target domains. Base…
Domain GeneralizationOut-of-Distribution GeneralizationTowards Transferable Targeted Adversarial Examples
Transferability of adversarial examples is critical for black-box deep learning model attacks. While most existing studies focus on enhancing the transferability of untargeted adversarial attacks, few of them studied…
Adversarial Attack