Generative Autoencoding of Dropout Patterns
We propose a generative model termed Deciphering Autoencoders. In this model, we assign a unique random dropout pattern to each data point in the training dataset and then train an autoencoder to reconstruct the corresponding data point using this pattern as information to be encoded. Even if a completely random dropout pattern is assigned to each data point regardless of their similarities, a sufficiently large encoder can smoothly map them to a low-dimensional latent space to reconstruct individual training data points. During inference, using a dropout pattern different from those used during training allows the model to function as a generator. Since the training of Deciphering Autoencoders relies solely on reconstruction error, it offers more stable training compared to other generative models. Despite their simplicity, Deciphering Autoencoders show sampling quality comparable to DCGAN on the CIFAR-10 dataset.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep late…
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks
Neural networks are often over-parameterized and hence benefit from aggressive regularization. Conventional regularization methods, such as Dropout or weight decay, do not leverage the structures of the network's inputs …
Image ClassificationLanguage ModelingLanguage ModellingMachine TranslationCoVAE: Consistency Training of Variational Autoencoders
Current state-of-the-art generative approaches frequently rely on a two-stage training procedure, where an autoencoder (often a VAE) first performs dimensionality reduction, followed by training a generative model on the…
Dimensionality ReductionAutoencoding Generative Adversarial Networks
In the years since Goodfellow et al. introduced Generative Adversarial Networks (GANs), there has been an explosion in the breadth and quality of generative model applications. Despite this work, GANs still have a long w…
Generative Adversarial NetworkSemi-supervised Autoencoding Projective Dependency Parsing
We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the input using continuous latent variables in…
DecoderDependency Parsing