A Deep and Tractable Density Estimator
The Neural Autoregressive Distribution Estimator (NADE) and its real-valued version RNADE are competitive density models of multidimensional data across a variety of domains. These models use a fixed, arbitrary ordering of the data dimensions. One can easily condition on variables at the beginning of the ordering, and marginalize out variables at the end of the ordering, however other inference tasks require approximate inference. In this work we introduce an efficient procedure to simultaneously train a NADE model for each possible ordering of the variables, by sharing parameters across all these models. We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available. Moreover, unlike the original NADE, our training procedure scales to deep models. Empirically, ensembles of Deep NADE models obtain state of the art density estimation performance.
Code (1)
Tasks
Density EstimationImage GenerationSimilar Papers 제목 키워드 기반
Fisher Efficient Inference of Intractable Models
Maximum Likelihood Estimators (MLE) has many good properties. For example, the asymptotic variance of MLE solution attains equality of the asymptotic Cram{\'e}r-Rao lower bound (efficiency bound), which is the minimum po…
Density Ratio EstimationRNADE: The real-valued neural autoregressive density-estimator
We introduce RNADE, a new model for joint density estimation of real-valued vectors. Our model calculates the density of a datapoint as the product of one-dimensional conditionals modeled using mixture density networks w…
Density EstimationEfficient Nonparametric Smoothness Estimation
Sobolev quantities (norms, inner products, and distances) of probability density functions are important in the theory of nonparametric statistics, but have rarely been used in practice, partly due to a lack of practical…
Two-sample testingFast Nonparametric Conditional Density Estimation
Conditional density estimation generalizes regression by modeling a full density f(yjx) rather than only the expected value E(yjx). This is important for many tasks, including handling multi-modality and generating predi…
Density EstimationPrediction IntervalsScore Matching for Truncated Density Estimation on a Manifold
When observations are truncated, we are limited to an incomplete picture of our dataset. Recent methods propose to use score matching for truncated density estimation, where the access to the intractable normalising cons…
Density Estimation