Generative Robust Classification
Training adversarially robust discriminative (i.e., softmax) classifier has been the dominant approach to robust classification. Building on recent work on adversarial training (AT)-based generative models, we investigate using AT to learn unnormalized class-conditional density models and then performing generative robust classification. Our result shows that, under the condition of similar model capacities, the generative robust classifier achieves comparable performance to a baseline softmax robust classifier when the test data is clean or when the test perturbation is of limited size, and much better performance when the test perturbation size exceeds the training perturbation size. The generative classifier is also able to generate samples or counterfactuals that more closely resemble the training data, suggesting that the generative classifier can better capture the class-conditional distributions. In contrast to standard discriminative adversarial training where advanced data augmentation techniques are only effective when combined with weight averaging, we find it straightforward to apply advanced data augmentation to achieve better robustness in our approach. Our result suggests that the generative classifier is a competitive alternative to robust classification, especially for problems with limited number of classes.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationData AugmentationRobust classificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On The Distribution of Penultimate Activations of Classification Networks
This paper studies probability distributions of penultimate activations of classification networks. We show that, when a classification network is trained with the cross-entropy loss, its final classification layer forms…
ClassificationConditional Image GenerationImage GenerationKnowledge DistillationGenerative and Discriminative Text Classification with Recurrent Neural Networks
We empirically characterize the performance of discriminative and generative LSTM models for text classification. We find that although RNN-based generative models are more powerful than their bag-of-words ancestors (e.g…
ClassificationContinual LearningGeneral Classificationtext-classification+1Distance Based Image Classification: A solution to generative classification's conundrum?
Most classifiers rely on discriminative boundaries that separate instances of each class from everything else. We argue that discriminative boundaries are counter-intuitive as they define semantics by what-they-are-not; …
image-classificationImage ClassificationFrom Graph Generation to Graph Classification
This note describes a new approach to classifying graphs that leverages graph generative models (GGM). Assuming a GGM that defines a joint probability distribution over graphs and their class labels, I derive classificat…
ClassificationGraph ClassificationGraph GenerationGenerative-Discriminative Variational Model for Visual Recognition
The paradigm shift from shallow classifiers with hand-crafted features to end-to-end trainable deep learning models has shown significant improvements on supervised learning tasks. Despite the promising power of deep neu…
ClassificationGeneral ClassificationmodelMulti-class Classification+3