Automatic Relevance Determination For Deep Generative Models
A recurring problem when building probabilistic latent variable models is regularization and model selection, for instance, the choice of the dimensionality of the latent space. In the context of belief networks with latent variables, this problem has been adressed with Automatic Relevance Determination (ARD) employing Monte Carlo inference. We present a variational inference approach to ARD for Deep Generative Models using doubly stochastic variational inference to provide fast and scalable learning. We show empirical results on a standard dataset illustrating the effects of contracting the latent space automatically. We show that the resulting latent representations are significantly more compact without loss of expressive power of the learned models.
Code (0)
등록된 구현이 없습니다.
Tasks
Model SelectionVariational InferenceSimilar Papers 제목 키워드 기반
Laplacian Prior Variational Automatic Relevance Determination for Transmission Tomography
In the classic sparsity-driven problems, the fundamental L-1 penalty method has been shown to have good performance in reconstructing signals for a wide range of problems. However this performance relies on a good choice…
An Automatic Relevance Determination Prior Bayesian Neural Network for Controlled Variable Selection
We present an Automatic Relevance Determination prior Bayesian Neural Network(BNN-ARD) weight l2-norm measure as a feature importance statistic for the model-x knockoff filter. We show on both simulated data and the Norw…
Feature ImportanceVariable SelectionVariable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution
Variable selection for Gaussian process models is often done using automatic relevance determination, which uses the inverse length-scale parameter of each input variable as a proxy for variable relevance. This implicitl…
Gaussian ProcessesSensitivityVariable SelectionEfficient Language Modeling with Automatic Relevance Determination in Recurrent Neural Networks
Reduction of the number of parameters is one of the most important goals in Deep Learning. In this article we propose an adaptation of Doubly Stochastic Variational Inference for Automatic Relevance Determination (DSVI-A…
DecoderLanguage ModelingLanguage ModellingVariational InferenceCorrentropy-Based Logistic Regression with Automatic Relevance Determination for Robust Sparse Brain Activity Decoding
Recent studies have utilized sparse classifications to predict categorical variables from high-dimensional brain activity signals to expose human's intentions and mental states, selecting the relevant features automatica…
Brain DecodingClassificationEEGElectroencephalogram (EEG)+1