Adversarial Robustness is at Odds with Lazy Training
Recent works show that adversarial examples exist for random neural networks [Daniely and Schacham, 2020] and that these examples can be found using a single step of gradient ascent [Bubeck et al., 2021]. In this work, we extend this line of work to "lazy training" of neural networks -- a dominant model in deep learning theory in which neural networks are provably efficiently learnable. We show that over-parametrized neural networks that are guaranteed to generalize well and enjoy strong computational guarantees remain vulnerable to attacks generated using a single step of gradient ascent.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessLearning TheorySimilar Papers 제목 키워드 기반
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes
Neural networks are known to be highly sensitive to adversarial examples. These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem. To better understand th…
Adversarial RobustnessEvolution of Neural Tangent Kernels under Benign and Adversarial Training
Two key challenges facing modern deep learning are mitigating deep networks' vulnerability to adversarial attacks and understanding deep learning's generalization capabilities. Towards the first issue, many defense strat…
Adversarial RobustnessHelper-based Adversarial Training: Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off
While adversarial training has become the de facto approach for training robust classifiers, it leads to a drop in accuracy. This has led to prior works postulating that accuracy is inherently at odds with robustness. Ye…
Adversarial DefenseAdversarial RobustnessRobust classificationReducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off
While adversarial training has become the de facto approach for training robust classifiers, it leads to a drop in accuracy. This has led to prior works postulating that accuracy is inherently at odds with robustness. Ye…
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)
We study the average robustness notion in deep neural networks in (selected) wide and narrow, deep and shallow, as well as lazy and non-lazy training settings. We prove that in the under-parameterized setting, width has …