Estimating Functions of Probability Distributions from a Finite Set of Samples, Part 1: Bayes Estimators and the Shannon Entropy
We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint distribution, we present finite sample estimators for the mutual information, covariance, and chi-squared functions of that probability distribution.
Code (1)
Similar Papers 제목 키워드 기반
Data-Driven Reachability analysis and Support set Estimation with Christoffel Functions
We present algorithms for estimating the forward reachable set of a dynamical system using only a finite collection of independent and identically distributed samples. The produced estimate is the sublevel set of a funct…
Sublinear Algorithms for Wasserstein and Total Variation Distances: Applications to Fairness and Privacy Auditing
Resource-efficiently computing representations of probability distributions and the distances between them while only having access to the samples is a fundamental and useful problem across mathematical sciences. In this…
FairnessFederated LearningUpper-Confidence-Bound Algorithms for Active Learning in Multi-Armed Bandits
In this paper, we study the problem of estimating uniformly well the mean values of several distributions given a finite budget of samples. If the variance of the distributions were known, one could design an optimal sam…
Active LearningMulti-Armed BanditsNeural Monge Map estimation and its applications
Monge map refers to the optimal transport map between two probability distributions and provides a principled approach to transform one distribution to another. Neural network based optimal transport map solver has gaine…
Image GenerationImage InpaintingText to Image GenerationText-to-Image GenerationOn the Estimation of Information Measures of Continuous Distributions
The estimation of information measures of continuous distributions based on samples is a fundamental problem in statistics and machine learning. In this paper, we analyze estimates of differential entropy in $K$-dimensio…