Universally Consistent Online Learning with Arbitrarily Dependent Responses
This work provides an online learning rule that is universally consistent under processes on (X,Y) pairs, under conditions only on the X process. As a special case, the conditions admit all processes on (X,Y) such that the process on X is stationary. This generalizes past results which required stationarity for the joint process on (X,Y), and additionally required this process to be ergodic. In particular, this means that ergodicity is superfluous for the purpose of universally consistent online learning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Universally consistent predictive distributions
This paper describes simple universally consistent procedures of probability forecasting that satisfy a natural property of small-sample validity, under the assumption that the observations are produced independently in …
A universally consistent learning rule with a universally monotone error
We present a universally consistent learning rule whose expected error is monotone non-increasing with the sample size under every data distribution. The question of existence of such rules was brought up in 1996 by Devr…
DiversityUniversal consistency of the $k$-NN rule in metric spaces and Nagata dimension. II
We continue to investigate the $k$ nearest neighbour ($k$-NN) learning rule in complete separable metric spaces. Thanks to the results of C\'erou and Guyader (2006) and Preiss (1983), this rule is known to be universally…
Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processes
This work initiates a general study of learning and generalization without the i.i.d. assumption, starting from first principles. While the traditional approach to statistical learning theory typically relies on standard…
Learning TheoryUniversal Online Learning: an Optimistically Universal Learning Rule
We study the subject of universal online learning with non-i.i.d. processes for bounded losses. The notion of an universally consistent learning was defined by Hanneke in an effort to study learning theory under minimal …
Learning TheoryMemorization