Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions. The new statistic avoids the explicit estimation of the underlying distributions in highdimensional space and it operates on the cone of symmetric positive semidefinite (SPS) matrix using the Bregman matrix divergence. Moreover, it inherits the merits of the correntropy function to explicitly incorporate high-order statistics in the data. We present the properties of our new statistic and illustrate its connections to prior art. We finally show the applications of our new statistic on three different machine learning problems, namely the multi-task learning over graphs, the concept drift detection, and the information-theoretic feature selection, to demonstrate its utility and advantage. Code of our statistic is available at https://bit.ly/BregmanCorrentropy.
Code (1)
Tasks
Drift Detectionfeature selectionMulti-Task LearningSimilar Papers 제목 키워드 기반
Measuring Differences between Conditional Distributions using Kernel Embeddings
Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddi…
Neural Characteristic Function Learning for Conditional Image Generation
The emergence of conditional generative adversarial networks (cGANs) has revolutionised the way we approach and control the generation, by means of adversarially learning joint distributions of data and auxiliary inf…
Conditional Image GenerationGenerative Adversarial NetworkImage GenerationComparing Distributions by Measuring Differences that Affect Decision Making
Measuring the discrepancy between two probability distributions is a fundamental problem in machine learning and statistics. We propose a new class of discrepancies based on the optimal loss for a decision task -- two di…
Decision MakingKernelized Complete Conditional Stein Discrepancy
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples fr…
A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation
We derive a new discrepancy statistic for measuring differences between two probability distributions based on combining Stein's identity with the reproducing kernel Hilbert space theory. We apply our result to test how …