Robust Learning with Jacobian Regularization
Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks. The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.
Code (3)
Tasks
BIG-bench Machine LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
Adversarially robust generalization theory via Jacobian regularization for deep neural networks
Powerful deep neural networks are vulnerable to adversarial attacks. To obtain adversarially robust models, researchers have separately developed adversarial training and Jacobian regularization techniques. There are abu…
Adversarial AttackGeneralizing and Improving Jacobian and Hessian Regularization
Jacobian and Hessian regularization aim to reduce the magnitude of the first and second-order partial derivatives with respect to neural network inputs, and they are predominantly used to ensure the adversarial robustnes…
Adversarial RobustnessConditioning of Reinforcement Learning Agents and its Policy Regularization Application
The outcome of Jacobian singular values regularization was studied for supervised learning problems. It also was shown that Jacobian conditioning regularization can help to avoid the ``mode-collapse'' problem in Generati…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+1Tangent-Space Regularization for Neural-Network Models of Dynamical Systems
This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exh…
Nuclear Norm Regularization for Deep Learning
Penalizing the nuclear norm of a function's Jacobian encourages it to locally behave like a low-rank linear map. Such functions vary locally along only a handful of directions, making the Jacobian nuclear norm a natural …
Deep LearningDenoisingRepresentation Learning