A Variational Inequality Perspective on Generative Adversarial Networks
Generative adversarial networks (GANs) form a generative modeling approach known for producing appealing samples, but they are notably difficult to train. One common way to tackle this issue has been to propose new formulations of the GAN objective. Yet, surprisingly few studies have looked at optimization methods designed for this adversarial training. In this work, we cast GAN optimization problems in the general variational inequality framework. Tapping into the mathematical programming literature, we counter some common misconceptions about the difficulties of saddle point optimization and propose to extend techniques designed for variational inequalities to the training of GANs. We apply averaging, extrapolation and a computationally cheaper variant that we call extrapolation from the past to the stochastic gradient method (SGD) and Adam.
Code (1)
Tasks
MisconceptionsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mirror Descent-Type Algorithms for the Variational Inequality Problem with Functional Constraints
Variational inequalities play a key role in machine learning research, such as generative adversarial networks, reinforcement learning, adversarial training, and generative models. This paper is devoted to the constraine…
Reinforcement LearningA game-theoretic approach for Generative Adversarial Networks
Generative adversarial networks (GANs) are a class of generative models, known for producing accurate samples. The key feature of GANs is that there are two antagonistic neural networks: the generator and the discriminat…
No-Regret Generative Modeling via Parabolic Monge-Ampère PDE
We introduce a novel generative modeling framework based on a discretized parabolic Monge-Amp\`ere PDE, which emerges as a continuous limit of the Sinkhorn algorithm commonly used in optimal transport. Our method perform…
Variational InferenceOptimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities
The optimization problems associated with training generative adversarial neural networks can be largely reduced to certain {\em non-monotone} variational inequality problems (VIPs), whereas existing convergence results …
Variational inequality for perpetual American option price and convergence to the solution of the difference equation
A variational inequality for pricing the perpetual American option and the corresponding difference equation are considered. First, the maximum principle and uniqueness of the solution to variational inequality for prici…