Theoretically Principled Trade-off for Stateful Defenses against Query-Based Black-Box Attacks
Adversarial examples threaten the integrity of machine learning systems with alarming success rates even under constrained black-box conditions. Stateful defenses have emerged as an effective countermeasure, detecting potential attacks by maintaining a buffer of recent queries and detecting new queries that are too similar. However, these defenses fundamentally pose a trade-off between attack detection and false positive rates, and this trade-off is typically optimized by hand-picking feature extractors and similarity thresholds that empirically work well. There is little current understanding as to the formal limits of this trade-off and the exact properties of the feature extractors/underlying problem domain that influence it. This work aims to address this gap by offering a theoretical characterization of the trade-off between detection and false positive rates for stateful defenses. We provide upper bounds for detection rates of a general class of feature extractors and analyze the impact of this trade-off on the convergence of black-box attacks. We then support our theoretical findings with empirical evaluations across multiple datasets and stateful defenses.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection
ML models are known to be vulnerable to adversarial query attacks. In these attacks, queries are iteratively perturbed towards a particular class without any knowledge of the target model besides its output. The prevalen…
Malware DetectionOut-of-Distribution DetectionStateful Defenses for Machine Learning Models Are Not Yet Secure Against Black-box Attacks
Recent work has proposed stateful defense models (SDMs) as a compelling strategy to defend against a black-box attacker who only has query access to the model, as is common for online machine learning platforms. Such sta…
Benchmarking Misuse Mitigation Against Covert Adversaries
Existing language model safety evaluations focus on overt attacks and low-stakes tasks. Realistic attackers can subvert current safeguards by requesting help on small, benign-seeming tasks across many independent queries…
BenchmarkingLanguage ModelingLanguage ModellingTheoretically Principled Trade-off between Robustness and Accuracy
We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples. Although this problem has been widely studied empirically, much remains u…
Adversarial AttackAdversarial DefenseAdversarial RobustnessGeneral ClassificationDecomposition Attacks Across Unlinkable Identities: Limits of Stateful Defenses for LLM Services
Most large language model services use stateless defenses, which judge only the current request, to refuse harmful tasks. Decomposition attacks exploit this limitation by splitting a harmful task into individually permis…