On the advantages of stochastic encoders
Stochastic encoders have been used in rate-distortion theory and neural compression because they can be easier to handle. However, in performance comparisons with deterministic encoders they often do worse, suggesting that noise in the encoding process may generally be a bad idea. It is poorly understood if and when stochastic encoders do better than deterministic encoders. In this paper we provide one illustrative example which shows that stochastic encoders can significantly outperform the best deterministic encoders. Our toy example suggests that stochastic encoders may be particularly useful in the regime of "perfect perceptual quality".
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Stochastic Ratio Matching of RBMs for Sparse High-Dimensional Inputs
Sparse high-dimensional data vectors are common in many application domains where a very large number of rarely non-zero features can be devised. Unfortunately, this creates a computational bottleneck for unsupervised fe…
text-classificationText ClassificationVocal Bursts Intensity PredictionFrom Variational to Deterministic Autoencoders
Variational Autoencoders (VAEs) provide a theoretically-backed and popular framework for deep generative models. However, learning a VAE from data poses still unanswered theoretical questions and considerable practical c…
DecoderDensity EstimationRobust Regression via Model Based Methods
The mean squared error loss is widely used in many applications, including auto-encoders, multi-target regression, and matrix factorization, to name a few. Despite computational advantages due to its differentiability, i…
modelMulti-target regressionregressionModel Selection for Bayesian Autoencoders
We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and i…
modelModel SelectionRepresentation LearningLearning Generative Factors of EEG Data with Variational auto-encoders
Electroencephalography produces high-dimensional, stochastic data from which it might be challenging to extract high-level knowledge about the phenomena of interest. We address this challenge by applying the framework of…
EEGElectroencephalogram (EEG)