Domain Adaptation for Efficiently Fine-tuning Vision Transformer with Encrypted Images

2023-09-05 · Teru Nagamori, Sayaka Shiota, Hitoshi Kiya

In recent years, deep neural networks (DNNs) trained with transformed data have been applied to various applications such as privacy-preserving learning, access control, and adversarial defenses. However, the use of transformed data decreases the performance of models. Accordingly, in this paper, we propose a novel method for fine-tuning models with transformed images under the use of the vision transformer (ViT). The proposed domain adaptation method does not cause the accuracy degradation of models, and it is carried out on the basis of the embedding structure of ViT. In experiments, we confirmed that the proposed method prevents accuracy degradation even when using encrypted images with the CIFAR-10 and CIFAR-100 datasets.

📄 PDF Abstract BibTeX arXiv:2309.02556

Code (0)

등록된 구현이 없습니다.

Tasks

Domain AdaptationPrivacy Preserving

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Vision Transformer The Vision Transformer, or ViT, is a model for image classification that employs a Transformer-like architecture over…

Efficient Fine-Tuning with Domain Adaptation for Privacy-Preserving Vision Transformer

2024-01-10 · Teru Nagamori, Sayaka Shiota, Hitoshi Kiya

We propose a novel method for privacy-preserving deep neural networks (DNNs) with the Vision Transformer (ViT). The method allows us not only to train models and test with visually protected images but to also avoid the …

Domain Adaptation image-classification Image Classification Privacy Preserving

VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models

2025-10-23 · Jesimon Barreto, Carlos Caetano, André Araujo, William Robson Schwartz arxiv

Foundation models have advanced computer vision by enabling strong performance across diverse tasks through large-scale pretraining and supervised fine-tuning. However, they may underperform in domains with distribution …

Self-Supervised Learning

Visual Prompt Tuning for Test-time Domain Adaptation

2022-10-10 · Yunhe Gao, Xingjian Shi, Yi Zhu, Hao Wang 외

Models should be able to adapt to unseen data during test-time to avoid performance drops caused by inevitable distribution shifts in real-world deployment scenarios. In this work, we tackle the practical yet challenging…

Domain Adaptation Test-time Adaptation Unsupervised Domain Adaptation Visual Prompt Tuning

Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models

2024-07-22 · Raza Imam, Hanan Gani, Muhammad Huzaifa, Karthik Nandakumar

The conventional modus operandi for adapting pre-trained vision-language models (VLMs) during test-time involves tuning learnable prompts, ie, test-time prompt tuning. This paper introduces Test-Time Low-rank adaptation …

Zero-shot Generalization

Effective Fine-Tuning of Vision Transformers with Low-Rank Adaptation for Privacy-Preserving Image Classification

2025-07-16 · Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya arxiv

We propose a low-rank adaptation method for training privacy-preserving vision transformer (ViT) models that efficiently freezes pre-trained ViT model weights. In the proposed method, trainable rank decomposition matrice…

Image Classification