Analytical Probability Distributions and Exact Expectation-Maximization for Deep Generative Networks
Deep Generative Networks (DGNs) with probabilistic modeling of their output and latent space are currently trained via Variational Autoencoders (VAEs). In the absence of a known analytical form for the posterior and likelihood expectation, VAEs resort to approximations, including (Amortized) Variational Inference (AVI) and Monte-Carlo sampling. We exploit the Continuous Piecewise Affine property of modern DGNs to derive their posterior and marginal distributions as well as the latter's first two moments. These findings enable us to derive an analytical Expectation-Maximization (EM) algorithm for gradient-free DGN learning. We demonstrate empirically that EM training of DGNs produces greater likelihood than VAE training. Our new framework will guide the design of new VAE AVI that better approximates the true posterior and open new avenues to apply standard statistical tools for model comparison, anomaly detection, and missing data imputation.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionImputationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Analytical Probability Distributions and EM-Learning for Deep Generative Networks
Deep Generative Networks (DGNs) with probabilistic modeling of their output and latent space are currently trained via Variational Autoencoders (VAEs). In the absence of a known analytical form for the posterior and like…
Anomaly DetectionImputationVariational InferenceSEMF: Supervised Expectation-Maximization Framework for Predicting Intervals
This work introduces the Supervised Expectation-Maximization Framework (SEMF), a versatile and model-agnostic approach for generating prediction intervals with any ML model. SEMF extends the Expectation-Maximization algo…
PredictionPrediction Intervalsquantile regressionregression+1Big Learning Expectation Maximization
Mixture models serve as one fundamental tool with versatile applications. However, their training techniques, like the popular Expectation Maximization (EM) algorithm, are notoriously sensitive to parameter initializatio…
Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality
By utilizing recently developed tools for constructing gradient flows on Wasserstein spaces, we extend an analysis technique commonly employed to understand alternating minimization algorithms on Euclidean space to the E…
Deep Generative Clustering with VAEs and Expectation-Maximization
We propose a novel deep clustering method that integrates Variational Autoencoders (VAEs) into the Expectation-Maximization (EM) framework. Our approach models the probability distribution of each cluster with a VAE and …
ClusteringDeep Clustering