Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement
Adversarial training (AT) is currently one of the most effective ways to obtain the robustness of deep neural networks against adversarial attacks. However, most AT methods suffer from robust overfitting, i.e., a significant generalization gap in adversarial robustness between the training and testing curves. In this paper, we first identify a connection between robust overfitting and the excessive memorization of noisy labels in AT from a view of gradient norm. As such label noise is mainly caused by a distribution mismatch and improper label assignments, we are motivated to propose a label refinement approach for AT. Specifically, our Self-Guided Label Refinement first self-refines a more accurate and informative label distribution from over-confident hard labels, and then it calibrates the training by dynamically incorporating knowledge from self-distilled models into the current model and thus requiring no external teachers. Empirical results demonstrate that our method can simultaneously boost the standard accuracy and robust performance across multiple benchmark datasets, attack types, and architectures. In addition, we also provide a set of analyses from the perspectives of information theory to dive into our method and suggest the importance of soft labels for robust generalization.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessMemorizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Deflecting Adversarial Attacks with Pixel Deflection
CNNs are poised to become integral parts of many critical systems. Despite their robustness to natural variations, image pixel values can be manipulated, via small, carefully crafted, imperceptible perturbations, to caus…
Adversarial AttackImproving Fast Adversarial Training via Self-Knowledge Guidance
Adversarial training has achieved remarkable advancements in defending against adversarial attacks. Among them, fast adversarial training (FAT) is gaining attention for its ability to achieve competitive robustness with …
Adversarial RobustnessAttributeDynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Models
Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) exhibit perceptual fragility when confronted with visually complex scenes. This weakness stems from a reliance on finite training datasets, …
Reinforcement LearningThe Attacker in the Mirror: Breaking Self-Consistency in Safety via Anchored Bipolicy Self-Play
Self-play red team is an established approach to improving AI safety in which different instances of the same model play attacker and defender roles in a zero-sum game, i.e., where the attacker tries to jailbreak the def…
Exposing and Defending the Achilles' Heel of Video Mixture-of-Experts
Mixture-of-Experts (MoE) has demonstrated strong performance in video understanding tasks, yet its adversarial robustness remains underexplored. Existing attack methods often treat MoE as a unified architecture, overlook…
Adversarial Robustness