The Adversarial Consistency of Surrogate Risks for Binary Classification
We study the consistency of surrogate risks for robust binary classification. It is common to learn robust classifiers by adversarial training, which seeks to minimize the expected $0$-$1$ loss when each example can be maliciously corrupted within a small ball. We give a simple and complete characterization of the set of surrogate loss functions that are \emph{consistent}, i.e., that can replace the $0$-$1$ loss without affecting the minimizing sequences of the original adversarial risk, for any data distribution. We also prove a quantitative version of adversarial consistency for the $\rho$-margin loss. Our results reveal that the class of adversarially consistent surrogates is substantially smaller than in the standard setting, where many common surrogates are known to be consistent.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationClassificationSimilar Papers 제목 키워드 기반
The Consistency of Adversarial Training for Binary Classification
Robustness to adversarial perturbations is of paramount concern in modern machine learning. One of the state-of-the-art methods for training robust classifiers is adversarial training, which involves minimizing a supremu…
BIG-bench Machine LearningBinary ClassificationClassificationExistence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification
Adversarial training is one of the most popular methods for training methods robust to adversarial attacks, however, it is not well-understood from a theoretical perspective. We prove and existence, regularity, and minim…
Adversarial RobustnessBinary ClassificationAdversarial Surrogate Risk Bounds for Binary Classification
A central concern in classification is the vulnerability of machine learning models to adversarial attacks. Adversarial training is one of the most popular techniques for training robust classifiers, which involves minim…
Binary ClassificationClassificationFundamental Novel Consistency Theory: $H$-Consistency Bounds
In machine learning, the loss functions optimized during training often differ from the target loss that defines task performance due to computational intractability or lack of differentiability. We present an in-depth s…
Multi-class ClassificationBinary ClassificationAdversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier
Minimizing an adversarial surrogate risk is a common technique for learning robust classifiers. Prior work showed that convex surrogate losses are not statistically consistent in the adversarial context -- or in other wo…
Classification