Information Theoretic Learning with Infinitely Divisible Kernels
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices. We formulate an entropy-like functional on positive definite matrices based on Renyi's axiomatic definition of entropy and examine some key properties of this functional that lead to the concept of infinite divisibility. The proposed formulation avoids the plug in estimation of density and brings along the representation power of reproducing kernel Hilbert spaces. As an application example, we derive a supervised metric learning algorithm using a matrix based analogue to conditional entropy achieving results comparable with the state of the art.
Code (0)
등록된 구현이 없습니다.
Tasks
Metric LearningSimilar Papers 제목 키워드 기반
Characteristic Kernels and Infinitely Divisible Distributions
We connect shift-invariant characteristic kernels to infinitely divisible distributions on $\mathbb{R}^{d}$. Characteristic kernels play an important role in machine learning applications with their kernel means to disti…
Measures of Entropy from Data Using Infinitely Divisible Kernels
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of informa…
ClusteringDimensionality ReductionTwo-sample testingInfinitely divisible privacy and beyond I: resolution of the $s^2=2k$ conjecture
Differential privacy is increasingly formalized through the lens of hypothesis testing via the robust and interpretable $f$-DP framework, where privacy guarantees are encoded by a baseline Blackwell trade-off function $f…
Infinitely Divisible Noise in the Low Privacy Regime
Federated learning, in which training data is distributed among users and never shared, has emerged as a popular approach to privacy-preserving machine learning. Cryptographic techniques such as secure aggregation are us…
Federated LearningPrivacy PreservingObvious manipulations, consistency, and the uniform rule
In the problem of fully allocating an infinitely divisible commodity among agents whose preferences are single-peaked, we show that the uniform rule is the only allocation rule that satisfies efficiency, the equal divisi…