Sum decomposition of divergence into three divergences
Divergence functions play a key role as to measure the discrepancy between two points in the field of machine learning, statistics and signal processing. Well-known divergences are the Bregman divergences, the Jensen divergences and the f-divergences. In this paper, we show that the symmetric Bregman divergence can be decomposed into the sum of two types of Jensen divergences and the Bregman divergence. Furthermore, applying this result, we show another sum decomposition of divergence is possible which includes f-divergences explicitly.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Bias-variance decompositions: the exclusive privilege of Bregman divergences
Bias-variance decompositions are widely used to understand the generalization performance of machine learning models. While the squared error loss permits a straightforward decomposition, other loss functions - such as z…
Efficient Bregman Range Search
We develop an algorithm for efficient range search when the notion of dissimilarity is given by a Bregman divergence. The range search task is to return all points in a potentially large database that are within some sp…
Density EstimationInformation RetrievalOutlier DetectionRetrievalCurved representational Bregman divergences and their applications
By analogy to curved exponential families in statistics, we define curved Bregman divergences as Bregman divergences restricted to nonlinear parameter subspaces. We show that the barycenter of a finite weighted set of pa…
Computing Marginal and Conditional Divergences between Decomposable Models with Applications
The ability to compute the exact divergence between two high-dimensional distributions is useful in many applications but doing so naively is intractable. Computing the alpha-beta divergence -- a family of divergences th…
A note on the quasiconvex Jensen divergences and the quasiconvex Bregman divergences derived thereof
We first introduce the class of strictly quasiconvex and strictly quasiconcave Jensen divergences which are oriented (asymmetric) distances, and study some of their properties. We then define the strictly quasiconvex Bre…