On Connections between Regularizations for Improving DNN Robustness
This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessBIG-bench Machine Learningimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
Using weight decay to penalize the L2 norms of weights in neural networks has been a standard training practice to regularize the complexity of networks. In this paper, we show that a family of regularizers, including we…
Adversarial RobustnessRecent Advances in Large Margin Learning
This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models f…
GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization
Deep learning is vulnerable to adversarial examples. Many defenses based on randomized neural networks have been proposed to solve the problem, but fail to achieve robustness against attacks using proxy gradients such as…
Adversarial RobustnessDiversityHuman Motion Detection Based on Dual-Graph and Weighted Nuclear Norm Regularizations
Motion detection has been widely used in many applications, such as surveillance and robotics. Due to the presence of the static background, a motion video can be decomposed into a low-rank background and a sparse foregr…
Motion DetectionMoving Object Detectionobject-detectionObject DetectionA Lifted $\ell_1 $ Framework for Sparse Recovery
Motivated by re-weighted $\ell_1$ approaches for sparse recovery, we propose a lifted $\ell_1$ (LL1) regularization which is a generalized form of several popular regularizations in the literature. By exploring such conn…