Adversarial Lipschitz Regularization

2019-07-12 · ICLR 2020 1 · Dávid Terjék

Generative adversarial networks (GANs) are one of the most popular approaches when it comes to training generative models, among which variants of Wasserstein GANs are considered superior to the standard GAN formulation in terms of learning stability and sample quality. However, Wasserstein GANs require the critic to be 1-Lipschitz, which is often enforced implicitly by penalizing the norm of its gradient, or by globally restricting its Lipschitz constant via weight normalization techniques. Training with a regularization term penalizing the violation of the Lipschitz constraint explicitly, instead of through the norm of the gradient, was found to be practically infeasible in most situations. Inspired by Virtual Adversarial Training, we propose a method called Adversarial Lipschitz Regularization, and show that using an explicit Lipschitz penalty is indeed viable and leads to competitive performance when applied to Wasserstein GANs, highlighting an important connection between Lipschitz regularization and adversarial training.

📄 PDF Abstract BibTeX arXiv:1907.05681

Code (2)

dterjek/adversarial_lipschitz_regularization pytorch

mbarbetti/tf-gen-models tf

Tasks

Image Generation

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Weight Normalization Weight Normalization is a normalization method for training neural networks. It is inspired by batch normalization,…

Dogecoin Customer Service Number +1-833-534-1729 설명 없음

Improved robustness to adversarial examples using Lipschitz regularization of the loss

2018-10-01 · ICLR 2019 5 · Chris Finlay, Adam Oberman, Bilal Abbasi

We augment adversarial training (AT) with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10. We obtain verifiabl…

Adversarial Robustness

Lipschitz regularized Deep Neural Networks generalize and are adversarially robust

2018-08-28 · Chris Finlay, Jeff Calder, Bilal Abbasi, Adam Oberman

In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does n…

Adversarial Robustness

Understanding Overfitting in Adversarial Training via Kernel Regression

2023-04-13 · Teng Zhang, Kang Li

Adversarial training and data augmentation with noise are widely adopted techniques to enhance the performance of neural networks. This paper investigates adversarial training and data augmentation with noise in the cont…

Data Augmentation regression

On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory

2020-06-15 · Alexandre Araujo, Benjamin Negrevergne, Yann Chevaleyre, Jamal Atif

This paper tackles the problem of Lipschitz regularization of Convolutional Neural Networks. Lipschitz regularity is now established as a key property of modern deep learning with implications in training stability, gene…

Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms

2023-09-21 · NeurIPS 2023 11

Multi-Agent Reinforcement Learning (MARL) has shown promising results across several domains. Despite this promise, MARL policies often lack robustness and are therefore sensitive to small changes in their environment. T…