Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examples
The vulnerability of artificial neural networks to adversarial perturbations in the black-box setting is widely studied in the literature. The majority of attack methods to construct these perturbations suffer from an impractically large number of queries required to find an adversarial example. In this work, we focus on knowledge distillation as an approach to conduct transfer-based black-box adversarial attacks and propose an iterative training of the surrogate model on an expanding dataset. This work is the first, to our knowledge, to provide provable guarantees on the success of knowledge distillation-based attack on classification neural networks: we prove that if the student model has enough learning capabilities, the attack on the teacher model is guaranteed to be found within the finite number of distillation iterations.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mutual Information Guided Backdoor Mitigation for Pre-trained Encoders
Self-supervised learning (SSL) is increasingly attractive for pre-training encoders without requiring labeled data. Downstream tasks built on top of those pre-trained encoders can achieve nearly state-of-the-art performa…
Knowledge DistillationSelf-Supervised LearningSELF-KNOWLEDGE DISTILLATION ADVERSARIAL ATTACK
Neural networks show great vulnerability under the threat of adversarial examples. By adding small perturbation to a clean image, neural networks with high classification accuracy can be completely fooled. One intr…
Adversarial AttackKnowledge DistillationSelf-Knowledge DistillationPay Attention to the Triggers: Constructing Backdoors That Survive Distillation
LLMs are often used by downstream users as teacher models for knowledge distillation, compressing their capabilities into memory-efficient models. However, as these teacher models may stem from untrusted parties, distill…
Knowledge DistillationContract And Conquer: How to Provably Compute Adversarial Examples for a Black-Box Model?
Black-box adversarial attacks are widely used as tools to test the robustness of deep neural networks against malicious perturbations of input data aimed at a specific change in the output of the model. Such methods, alt…
Knowledge DistillationMeta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
Pre-trained language models have been applied to various NLP tasks with considerable performance gains. However, the large model sizes, together with the long inference time, limit the deployment of such models in real-t…
Knowledge DistillationLanguage ModelingLanguage ModellingMeta-Learning+2