Density Estimation using Entropy Maximization for Semi-continuous Data
Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, we present a novel algorithm to estimate the density function for semi-continuous data using the principle of maximum entropy. Unlike existing methods in the literature, our algorithm needs only the sample values of the constraint functions in the entropy maximization problem and does not need the entire sample. Using simulations, we show that the estimate of the entropy produced by our algorithm has significantly less bias compared to existing methods. An application to the daily rainfall data is provided.
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationSimilar Papers 제목 키워드 기반
On the Estimation of Information Measures of Continuous Distributions
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensio…
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable. In this paper, we propose the amortized residual denoisin…
continuous-controlContinuous ControlDenoisingDensity EstimationLearning Mixture Density via Natural Gradient Expectation Maximization
Mixture density networks are neural networks that produce Gaussian mixtures to represent continuous multimodal conditional densities. Standard training procedures involve maximum likelihood estimation using the negative …
MESSY Estimation: Maximum-Entropy based Stochastic and Symbolic densitY Estimation
We introduce MESSY estimation, a Maximum-Entropy based Stochastic and Symbolic densitY estimation method. The proposed approach recovers probability density functions symbolically from samples using moments of a Gradient…
Density EstimationSymbolic RegressionThe Maximum von Neumann Entropy Principle: Theory and Applications in Machine Learning
Von Neumann entropy (VNE) is a fundamental quantity in quantum information theory and has recently been adopted in machine learning as a spectral measure of diversity for kernel matrices and kernel covariance operators. …