Adversarial Robustness as a Prior for Learned Representations
An important goal in deep learning is to learn versatile, high-level feature representations of input data. However, standard networks' representations seem to possess shortcomings that, as we illustrate, prevent them from fully realizing this goal. In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks. It turns out that representations learned by robust models address the aforementioned shortcomings and make significant progress towards learning a high-level encoding of inputs. In particular, these representations are approximately invertible, while allowing for direct visualization and manipulation of salient input features. More broadly, our results indicate adversarial robustness as a promising avenue for improving learned representations. Our code and models for reproducing these results is available at https://git.io/robust-reps .
Code (5)
Tasks
Adversarial RobustnessSimilar Papers 제목 키워드 기반
Metric Learning for Adversarial Robustness
Deep networks are well-known to be fragile to adversarial attacks. We conduct an empirical analysis of deep representations under the state-of-the-art attack method called PGD, and find that the attack causes the interna…
Adversarial RobustnessMetric LearningTowards Robust Graph Contrastive Learning
We study the problem of adversarially robust self-supervised learning on graphs. In the contrastive learning framework, we introduce a new method that increases the adversarial robustness of the learned representations t…
Adversarial RobustnessContrastive LearningSelf-Supervised LearningAdversarial Robustness for Unsupervised Domain Adaptation
Extensive Unsupervised Domain Adaptation (UDA) studies have shown great success in practice by learning transferable representations across a labeled source domain and an unlabeled target domain with deep models. However…
Adversarial RobustnessDomain AdaptationUnsupervised Domain AdaptationAdversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks
Adversarial training has been proven to be an effective technique for improving the adversarial robustness of models. However, there seems to be an inherent trade-off between optimizing the model for accuracy and robustn…
Adversarial RobustnessRobustness via Deep Low-Rank Representations
We investigate the effect of the dimensionality of the representations learned in Deep Neural Networks (DNNs) on their robustness to input perturbations, both adversarial and random. To achieve low dimensionality of lear…
ClusteringGeneral ClassificationImage ClassificationTransfer Learning