Improved learning theory for kernel distribution regression with two-stage sampling
The distribution regression problem encompasses many important statistics and machine learning tasks, and arises in a large range of applications. Among various existing approaches to tackle this problem, kernel methods have become a method of choice. Indeed, kernel distribution regression is both computationally favorable, and supported by a recent learning theory. This theory also tackles the two-stage sampling setting, where only samples from the input distributions are available. In this paper, we improve the learning theory of kernel distribution regression. We address kernels based on Hilbertian embeddings, that encompass most, if not all, of the existing approaches. We introduce the novel near-unbiased condition on the Hilbertian embeddings, that enables us to provide new error bounds on the effect of the two-stage sampling, thanks to a new analysis. We show that this near-unbiased condition holds for three important classes of kernels, based on optimal transport and mean embedding. As a consequence, we strictly improve the existing convergence rates for these kernels. Our setting and results are illustrated by numerical experiments.
Code (1)
Tasks
Learning TheoryregressionSimilar Papers 제목 키워드 기반
Out-of-Distribution Generalization in Kernel Regression
In real word applications, data generating process for training a machine learning model often differs from what the model encounters in the test stage. Understanding how and whether machine learning models generalize un…
BIG-bench Machine LearningOut-of-Distribution GeneralizationregressionTwo-stage Sampled Learning Theory on Distributions
We focus on the distribution regression problem: regressing to a real-valued response from a probability distribution. Although there exist a large number of similarity measures between distributions, very little is know…
Density EstimationLearning TheoryOpen-Ended Question Answeringregression+1Learning Theory for Distribution Regression
We focus on the distribution regression problem: regressing to vector-valued outputs from probability measures. Many important machine learning and statistical tasks fit into this framework, including multi-instance lear…
Density EstimationLearning TheoryOpen-Ended Question AnsweringregressionRobust Kernel-based Distribution Regression
Regularization schemes for regression have been widely studied in learning theory and inverse problems. In this paper, we study distribution regression (DR) which involves two stages of sampling, and aims at regressing f…
Learning TheoryregressionEstimates on Learning Rates for Multi-Penalty Distribution Regression
This paper is concerned with functional learning by utilizing two-stage sampled distribution regression. We study a multi-penalty regularization algorithm for distribution regression under the framework of learning theor…
Learning Theoryregression