Mixture Density Network Estimation of Continuous Variable Maximum Likelihood Using Discrete Training Samples
Mixture Density Networks (MDNs) can be used to generate probability density functions of model parameters $\boldsymbol{\theta}$ given a set of observables $\mathbf{x}$. In some applications, training data are available only for discrete values of a continuous parameter $\boldsymbol{\theta}$. In such situations a number of performance-limiting issues arise which can result in biased estimates. We demonstrate the usage of MDNs for parameter estimation, discuss the origins of the biases, and propose a corrective method for each issue.
Code (1)
Tasks
parameter estimationSimilar Papers 제목 키워드 기반
Learning Mixture Density via Natural Gradient Expectation Maximization
Mixture density networks are neural networks that produce Gaussian mixtures to represent continuous multimodal conditional densities. Standard training procedures involve maximum likelihood estimation using the negative …
Estimation of Information Theoretic Measures for Continuous Random Variables
We analyze the estimation of information theoretic measures of continuous random variables such as: differential entropy, mutual information or Kullback-Leibler divergence. The objective of this paper is two-fold. First,…
Density EstimationDensity Estimation using Entropy Maximization for Semi-continuous Data
Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, …
Density EstimationHistogram Meets Topic Model: Density Estimation by Mixture of Histograms
The histogram method is a powerful non-parametric approach for estimating the probability density function of a continuous variable. But the construction of a histogram, compared to the parametric approaches, demands a l…
Density EstimationEM Approaches to Nonparametric Estimation for Mixture of Linear Regressions
In a mixture of linear regression model, the regression coefficients are treated as random vectors that may follow either a continuous or discrete distribution. We propose two Expectation-Maximization (EM) algorithms to …