Lipschitz regularized Deep Neural Networks generalize
We show that if the usual training loss is augmented by a Lipschitz regularization term, then the networks generalize. We prove generalization by first establishing a stronger convergence result, along with a rate of convergence. A second result resolves a question posed in Zhang et al. (2016): how can a model distinguish between the case of clean labels, and randomized labels? Our answer is that Lipschitz regularization using the Lipschitz constant of the clean data makes this distinction. In this case, the model learns a different function which we hypothesize correctly fails to learn the dirty labels.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exactly Computing the Local Lipschitz Constant of ReLU Networks
The local Lipschitz constant of a neural network is a useful metric with applications in robustness, generalization, and fairness evaluation. We provide novel analytic results relating the local Lipschitz constant of non…
FairnessLipschitz regularized Deep Neural Networks generalize and are adversarially robust
In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does n…
Adversarial RobustnessLoss-Sensitive Generative Adversarial Networks on Lipschitz Densities
In this paper, we present the Lipschitz regularization theory and algorithms for a novel Loss-Sensitive Generative Adversarial Network (LS-GAN). Specifically, it trains a loss function to distinguish between real and fak…
Generative Adversarial Networkimage-classificationImage ClassificationImage GenerationDecentralized Feature-Distributed Optimization for Generalized Linear Models
We consider the "all-for-one" decentralized learning problem for generalized linear models. The features of each sample are partitioned among several collaborating agents in a connected network, but only one agent observ…
Distributed OptimizationStability of Image-Reconstruction Algorithms
Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularizat…
Image Reconstruction