Latent Bernoulli Autoencoder
In this work, we pose a question whether it is possible to design and train an autoencoder model in an end-to-end fashion to learn latent representations in multivariate Bernoulli space, and achieve performance comparable with the current state-of-the-art variational methods. Moreover, we investigate how to generate novel samples and perform smooth interpolation in the binary latent space. To meet our objective, we propose a simplified deterministic model with a straight-through estimator to learn the binary latents and show its competitiveness with the latest VAE methods. Furthermore, we propose a novel method based on a random hyperplane rounding for sampling and smooth interpolation in the multivariate Bernoulli latent space. Although not a main objective, we demonstrate that our methods perform on par or better than the current state-of-the-art methods on common CelebA, CIFAR-10 and MNIST datasets. PyTorch code and trained models to reproduce published results will be released with the camera ready version.
Code (1)
Similar Papers 제목 키워드 기반
The continuous Bernoulli: fixing a pervasive error in variational autoencoders
Variational autoencoders (VAE) have quickly become a central tool in machine learning, applicable to a broad range of data types and latent variable models. By far the most common first step, taken by seminal papers and …
Binary Noise for Binary Tasks: Masked Bernoulli Diffusion for Unsupervised Anomaly Detection
The high performance of denoising diffusion models for image generation has paved the way for their application in unsupervised medical anomaly detection. As diffusion-based methods require a lot of GPU memory and have l…
Anomaly DetectionDenoisingGPUImage Generation+1A lower bound for the ELBO of the Bernoulli Variational Autoencoder
We consider a variational autoencoder (VAE) for binary data. Our main innovations are an interpretable lower bound for its training objective, a modified initialization and architecture of such a VAE that leads to faster…
Fully Spiking Variational Autoencoder
Spiking neural networks (SNNs) can be run on neuromorphic devices with ultra-high speed and ultra-low energy consumption because of their binary and event-driven nature. Therefore, SNNs are expected to have various appli…
Image GenerationTime SeriesTime Series AnalysisAnomaly Detection in Aeronautics Data with Quantum-compatible Discrete Deep Generative Model
Deep generative learning cannot only be used for generating new data with statistical characteristics derived from input data but also for anomaly detection, by separating nominal and anomalous instances based on their r…
Anomaly DetectionTime Series