SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions

2020-05-29 · ACL 2020 6 · Mao Ye, Chengyue Gong, Qiang Liu

State-of-the-art NLP models can often be fooled by human-unaware transformations such as synonymous word substitution. For security reasons, it is of critical importance to develop models with certified robustness that can provably guarantee that the prediction is can not be altered by any possible synonymous word substitution. In this work, we propose a certified robust method based on a new randomized smoothing technique, which constructs a stochastic ensemble by applying random word substitutions on the input sentences, and leverage the statistical properties of the ensemble to provably certify the robustness. Our method is simple and structure-free in that it only requires the black-box queries of the model outputs, and hence can be applied to any pre-trained models (such as BERT) and any types of models (world-level or subword-level). Our method significantly outperforms recent state-of-the-art methods for certified robustness on both IMDB and Amazon text classification tasks. To the best of our knowledge, we are the first work to achieve certified robustness on large systems such as BERT with practically meaningful certified accuracy.

📄 PDF Abstract BibTeX arXiv:2005.14424

Code (1)

lushleaf/Structure-free-certified-NLP 공식 구현 pytorch

Tasks

text-classification Text Classification

Methods 이 논문이 사용한 방법론

Randomized Smoothing 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Weight Decay 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Adam 설명 없음

Multi-Head Attention 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

Certified vs. Empirical Adversarial Robust-ness via Hybrid Convolutions with Attention Stochasticity

2026-05-02 · Joy Dhar, Song Xia, Manish Kumar Pandey, Maryam Haghighat 외 arxiv

We introduce Hybrid Convolutions with Attention Stochasticity (HyCAS), an adversarial defense that narrows the long-standing gap between provable robustness under L2 certificates and empirical robustness against strong L…

Adversarial Robustness Adversarial Defense

(Certified!!) Adversarial Robustness for Free!

2022-06-21 · Nicholas Carlini, Florian Tramer, Krishnamurthy Dj Dvijotham, Leslie Rice 외

In this paper we show how to achieve state-of-the-art certified adversarial robustness to 2-norm bounded perturbations by relying exclusively on off-the-shelf pretrained models. To do so, we instantiate the denoised smoo…

Adversarial Robustness Denoising

Robustness of Unsupervised Representation Learning without Labels

2022-10-08 · Aleksandar Petrov, Marta Kwiatkowska

Unsupervised representation learning leverages large unlabeled datasets and is competitive with supervised learning. But non-robust encoders may affect downstream task robustness. Recently, robust representation encoders…

Representation Learning

Detection as Regression: Certified Object Detection with Median Smoothing

2020-12-01 · NeurIPS 2020 12 · Ping-Yeh Chiang, Michael Curry, Ahmed Abdelkader, Aounon Kumar 외

Despite the vulnerability of object detectors to adversarial attacks, very few defenses are known to date. While adversarial training can improve the empirical robustness of image classifiers, a direct extension to objec…

Object object-detection Object Detection regression

Detection as Regression: Certified Object Detection by Median Smoothing

2020-07-07 · Ping-Yeh Chiang, Michael J. Curry, Ahmed Abdelkader, Aounon Kumar 외