Concentration inequalities for leave-one-out cross validation
In this article we prove that estimator stability is enough to show that leave-one-out cross validation is a sound procedure, by providing concentration bounds in a general framework. In particular, we provide concentration bounds beyond Lipschitz continuity assumptions on the loss or on the estimator. We obtain our results by relying on random variables with distribution satisfying the logarithmic Sobolev inequality, providing us a relatively rich class of distributions. We illustrate our method by considering several interesting examples, including linear regression, kernel density estimation, and stabilized/truncated estimators such as stabilized kernel regression.
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationregressionSimilar Papers 제목 키워드 기반
Concentration Inequalities for Bounded Random Vectors
We derive simple concentration inequalities for bounded random vectors, which generalize Hoeffding's inequalities for bounded scalar random variables. As applications, we apply the general results to multinomial and Diri…
Introduction to Concentration Inequalities
In this report, we aim to exemplify concentration inequalities and provide easy to understand proofs for it. Our focus is on the inequalities which are helpful in the design and analysis of machine learning algorithms.
BIG-bench Machine LearningConcentration Inequalities for Multinoulli Random Variables
We investigate concentration inequalities for Dirichlet and Multinomial random variables.
Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space
We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical syste…
reinforcement-learningReinforcement Learning (RL)Statistical Agnostic Mapping: a Framework in Neuroimaging based on Concentration Inequalities
In the 70s a novel branch of statistics emerged focusing its effort in selecting a function in the pattern recognition problem, which fulfils a definite relationship between the quality of the approximation and its compl…
Two-sample testing