Patch Similarity Aware Data-Free Quantization for Vision Transformers

2022-03-04 · Zhikai Li, Liping Ma, Mengjuan Chen, Junrui Xiao, Qingyi Gu

Vision transformers have recently gained great success on various computer vision tasks; nevertheless, their high model complexity makes it challenging to deploy on resource-constrained devices. Quantization is an effective approach to reduce model complexity, and data-free quantization, which can address data privacy and security concerns during model deployment, has received widespread interest. Unfortunately, all existing methods, such as BN regularization, were designed for convolutional neural networks and cannot be applied to vision transformers with significantly different model architectures. In this paper, we propose PSAQ-ViT, a Patch Similarity Aware data-free Quantization framework for Vision Transformers, to enable the generation of "realistic" samples based on the vision transformer's unique properties for calibrating the quantization parameters. Specifically, we analyze the self-attention module's properties and reveal a general difference (patch similarity) in its processing of Gaussian noise and real images. The above insights guide us to design a relative value metric to optimize the Gaussian noise to approximate the real images, which are then utilized to calibrate the quantization parameters. Extensive experiments and ablation studies are conducted on various benchmarks to validate the effectiveness of PSAQ-ViT, which can even outperform the real-data-driven methods. Code is available at: https://github.com/zkkli/PSAQ-ViT.

📄 PDF Abstract BibTeX arXiv:2203.02250

Code (1)

zkkli/psaq-vit 공식 구현 pytorch

Tasks

Data Free Quantization Quantization

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers

2022-09-13 · Zhikai Li, Mengjuan Chen, Junrui Xiao, Qingyi Gu

Data-free quantization can potentially address data privacy and security concerns in model compression, and thus has been widely investigated. Recently, PSAQ-ViT designs a relative value metric, patch similarity, to gene…

Data Free Quantization image-classification Image Classification Model Compression+4

TriQDef: Disrupting Semantic and Gradient Alignment to Prevent Adversarial Patch Transferability in Quantized Neural Networks

2025-08-16 · Amira Guesmi, Bassem Ouni, Muhammad Shafique arxiv

Quantized Neural Networks (QNNs) are increasingly deployed in edge and resource-constrained environments due to their efficiency in computation and memory usage. While shown to distort the gradient landscape and weaken c…

Q-resafe: Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models

2025-06-25 · KeJia Chen, Jiawen Zhang, Jiacong Hu, Yu Wang 외

Quantized large language models (LLMs) have gained increasing attention and significance for enabling deployment in resource-constrained environments. However, emerging studies on a few calibration dataset-free quantizat…

Quantization

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

2026-03-20 · Xiaoming Yu, Shize Tang, Guanghua Yu, Linchuan Xie 외 arxiv

We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but…

MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity

2024-07-29 · Kanghyun Choi, Hye Yoon Lee, Dain Kwon, Sunjong Park 외

Data-free quantization (DFQ) is a technique that creates a lightweight network from its full-precision counterpart without the original training data, often through a synthetic dataset. Although several DFQ methods have …