Certified Patch Robustness via Smoothed Vision Transformers

2021-10-11 · CVPR 2022 1 · Hadi Salman, Saachi Jain, Eric Wong, Aleksander Mądry

Certified patch defenses can guarantee robustness of an image classifier to arbitrary changes within a bounded contiguous region. But, currently, this robustness comes at a cost of degraded standard accuracies and slower inference times. We demonstrate how using vision transformers enables significantly better certified patch robustness that is also more computationally efficient and does not incur a substantial drop in standard accuracy. These improvements stem from the inherent ability of the vision transformer to gracefully handle largely masked images. Our code is available at https://github.com/MadryLab/smoothed-vit.

📄 PDF Abstract BibTeX arXiv:2110.07719

Code (1)

madrylab/smoothed-vit 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Residual Connection 설명 없음

Multi-Head Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Vision Transformer The Vision Transformer, or ViT, is a model for image classification that employs a Transformer-like architecture over…

Towards Practical Certifiable Patch Defense with Vision Transformer

2022-03-16 · CVPR 2022 1 · Zhaoyu Chen, Bo Li, Jianghe Xu, Shuang Wu 외

Patch attacks, one of the most threatening forms of physical attack in adversarial examples, can lead networks to induce misclassification by modifying pixels arbitrarily in a continuous region. Certifiable patch defense…

Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness of Smoothed Classifiers

2025-03-21 · Gaojie Jin, Tianjin Huang, Ronghui Mu, Xiaowei Huang

Recent studies have identified a critical challenge in deep neural networks (DNNs) known as ``robust fairness", where models exhibit significant disparities in robust accuracy across different classes. While prior work h…

Adversarial Robustness Fairness

CertViT: Certified Robustness of Pre-Trained Vision Transformers

2023-02-01 · Kavya Gupta, Sagar Verma

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks…

Adversarial Robustness

Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifier

2025-09-30 · Gaojie Jin, Xinping Yi, Xiaowei Huang arxiv

Within the PAC-Bayesian framework, the Gibbs classifier (defined on a posterior $Q$) and the corresponding $Q$-weighted majority vote classifier are commonly used to analyze the generalization performance. However, there…

PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing

2021-11-19 · Yuheng Huang, Lei Ma, Yuanchun Li

Vision Transformer (ViT) is known to be highly nonlinear like other classical neural networks and could be easily fooled by both natural and adversarial patch perturbations. This limitation could pose a threat to the dep…