Defensive Dual Masking for Robust Adversarial Defense

2024-12-10 · Wangli Yang, Jie Yang, Yi Guo, Johan Barthelemy

The field of textual adversarial defenses has gained considerable attention in recent years due to the increasing vulnerability of natural language processing (NLP) models to adversarial attacks, which exploit subtle perturbations in input text to deceive models. This paper introduces the Defensive Dual Masking (DDM) algorithm, a novel approach designed to enhance model robustness against such attacks. DDM utilizes a unique adversarial training strategy where [MASK] tokens are strategically inserted into training samples to prepare the model to handle adversarial perturbations more effectively. During inference, potentially adversarial tokens are dynamically replaced with [MASK] tokens to neutralize potential threats while preserving the core semantics of the input. The theoretical foundation of our approach is explored, demonstrating how the selective masking mechanism strengthens the model's ability to identify and mitigate adversarial manipulations. Our empirical evaluation across a diverse set of benchmark datasets and attack mechanisms consistently shows that DDM outperforms state-of-the-art defense techniques, improving model accuracy and robustness. Moreover, when applied to Large Language Models (LLMs), DDM also enhances their resilience to adversarial attacks, providing a scalable defense mechanism for large-scale NLP applications.

📄 PDF Abstract BibTeX arXiv:2412.07078

Code (0)

등록된 구현이 없습니다.

Tasks

Adversarial Defense

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Random Gradient Masking as a Defensive Measure to Deep Leakage in Federated Learning

2024-08-15 · Joon Kim, Sejin Park

Federated Learning(FL), in theory, preserves privacy of individual clients' data while producing quality machine learning models. However, attacks such as Deep Leakage from Gradients(DLG) severely question the practicali…

Federated Learning

A Learning and Masking Approach to Secure Learning

2017-09-13 · Linh Nguyen, Sky Wang, Arunesh Sinha

Deep Neural Networks (DNNs) have been shown to be vulnerable against adversarial examples, which are data points cleverly constructed to fool the classifier. Such attacks can be devastating in practice, especially as DNN…

Autonomous Driving

Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

2018-09-13 · Siyue Wang, Xiao Wang, Pu Zhao, Wujie Wen 외

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as …

Defensive Few-shot Learning

2019-11-16 · Wenbin Li, Lei Wang, Xingxing Zhang, Lei Qi 외

This paper investigates a new challenging problem called defensive few-shot learning in order to learn a robust few-shot model against adversarial attacks. Simply applying the existing adversarial defense methods to few-…

Adversarial Defense Few-Shot Learning

A Low-Rank Defense Method for Adversarial Attack on Diffusion Models

2026-02-10 · Jiaxuan Zhu, Siyu Huang arxiv

Recently, adversarial attacks for diffusion models as well as their fine-tuning process have been developed rapidly. To prevent the abuse of these attack algorithms from affecting the practical application of diffusion m…

Adversarial Attack