Density Estimation via Discrepancy
Given i.i.d samples from some unknown continuous density on hyper-rectangle $[0, 1]^d$, we attempt to learn a piecewise constant function that approximates this underlying density non-parametrically. Our density estimate is defined on a binary split of $[0, 1]^d$ and built up sequentially according to discrepancy criteria; the key ingredient is to control the discrepancy adaptively in each sub-rectangle to achieve overall bound. We prove that the estimate, even though simple as it appears, preserves most of the estimation power. By exploiting its structure, it can be directly applied to some important pattern recognition tasks such as mode seeking and density landscape exploration. We demonstrate its applicability through simulations and examples.
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationSimilar Papers 제목 키워드 기반
Density estimation via mixture discrepancy and moments
With the aim of generalizing histogram statistics to higher dimensional cases, density estimation via discrepancy based sequential partition (DSP) has been proposed [D. Li, K. Yang, W. Wong, Advances in Neural Informatio…
Density EstimationDensity Estimation via Discrepancy Based Adaptive Sequential Partition
Given $iid$ observations from an unknown absolute continuous distribution defined on some domain $\Omega$, we propose a nonparametric method to learn a piecewise constant function to approximate the underlying probabilit…
Density EstimationApproximate Stein Classes for Truncated Density Estimation
Estimating truncated density models is difficult, as these models have intractable normalising constants and hard to satisfy boundary conditions. Score matching can be adapted to solve the truncated density estimation pr…
Density EstimationDiscrepancy, Coresets, and Sketches in Machine Learning
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrep…
BIG-bench Machine LearningDensity EstimationEstimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression
Functions of the ratio of the densities $p/q$ are widely used in machine learning to quantify the discrepancy between the two distributions $p$ and $q$. For high-dimensional distributions, binary classification-based den…
Binary ClassificationDensity Ratio EstimationMulti-class ClassificationMutual Information Estimation+2