Moment-Matching Conditions for Exponential Families with Conditioning or Hidden Data
Maximum likelihood learning with exponential families leads to moment-matching of the sufficient statistics, a classic result. This can be generalized to conditional exponential families and/or when there are hidden data. This document gives a first-principles explanation of these generalized moment-matching conditions, along with a self-contained derivation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A General Recipe for the Analysis of Randomized Multi-Armed Bandit Algorithms
In this paper we propose a general methodology to derive regret bounds for randomized multi-armed bandit algorithms. It consists in checking a set of sufficient conditions on the sampling probability of each arm and on t…
Thompson SamplingFinite Sample Bounds for Learning with Score Matching
Learning of continuous exponential family distributions with unbounded support remains an important area of research for both theory and applications in high-dimensional statistics. In recent years, score matching has be…
Improved MDL Estimators Using Fiber Bundle of Local Exponential Families for Non-exponential Families
Minimum Description Length (MDL) estimators, using two-part codes for universal coding, are analyzed. For general parametric families under certain regularity conditions, we introduce a two-part code whose regret is clos…
Sparse Continuous Distributions and Fenchel-Young Losses
Exponential families are widely used in machine learning, including many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical distributions via the softmax transformation)…
Audio ClassificationQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Exponential families from a single KL identity
Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions -- and underlie the theory of variational inference, entropy-regularized reinforcemen…
Reinforcement Learning