The Information-Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Modeling

2018-01-01 · ICLR 2018 1 · Shengjia Zhao, Jiaming Song, Stefano Ermon

A variety of learning objectives have been recently proposed for training generative models. We show that many of them, including InfoGAN, ALI/BiGAN, ALICE, CycleGAN, VAE, $\beta$-VAE, adversarial autoencoders, AVB, and InfoVAE, are Lagrangian duals of the same primal optimization problem. This generalization reveals the implicit modeling trade-offs between flexibility and computational requirements being made by these models. Furthermore, we characterize the class of all objectives that can be optimized under certain computational constraints. Finally, we show how this new Lagrangian perspective can explain undesirable behavior of existing methods and provide new principled solutions.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Batch Normalization 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Beta-VAE Beta-VAE is a type of variational autoencoder that seeks to discover disentangled latent factors. It modifies VAEs with an adjustable…

GAN Least Squares Loss GAN Least Squares Loss is a least squares loss function for generative adversarial networks. Minimizing this objective function is equivalent to minimizing the Pearson…

Cycle Consistency Loss Cycle Consistency Loss is a type of loss used for generative adversarial networks that performs unpaired image-to-image translation. It was introduced with the…

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models

2018-06-18 · Shengjia Zhao, Jiaming Song, Stefano Ermon

A large number of objectives have been proposed to train latent variable generative models. We show that many of them are Lagrangian dual functions of the same primal optimization problem. The primal problem optimizes th…

A Variational Perspective on Accelerated Methods in Optimization

2016-03-14 · Andre Wibisono, Ashia C. Wilson, Michael. I. Jordan

Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is n…

Structured Neural Topic Models for Reviews

2018-12-12 · Babak Esmaeili, Hongyi Huang, Byron C. Wallace, Jan-Willem van de Meent

We present Variational Aspect-based Latent Topic Allocation (VALTA), a family of autoencoding topic models that learn aspect-based representations of reviews. VALTA defines a user-item encoder that maps bag-of-words vect…

Sentence Topic Models

Towards Deeper Understanding of Variational Autoencoding Models

2017-02-28 · Shengjia Zhao, Jiaming Song, Stefano Ermon

We propose a new family of optimization criteria for variational auto-encoding models, generalizing the standard evidence lower bound. We provide conditions under which they recover the data distribution and learn latent…

Semi-supervised Autoencoding Projective Dependency Parsing

2020-11-02 · COLING 2020 8 · Xiao Zhang, Dan Goldwasser

We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the input using continuous latent variables in…

Decoder Dependency Parsing