Instance-dependent uniform tail bounds for empirical processes
We formulate a uniform tail bound for empirical processes indexed by a class of functions, in terms of the individual deviations of the functions rather than the worst-case deviation in the considered class. The tail bound is established by introducing an initial `deflation'' step to the standard generic chaining argument. The resulting tail bound is the sum of the complexity of the deflated function class'' in terms of a generalization of Talagrand's $\gamma$ functional, and the deviation of the function instance, both of which are formulated based on the natural seminorm induced by the corresponding Cram\'{e}r functions. Leveraging another less demanding natural seminorm, we also show similar bounds, though with implicit dependence on the sample size, in the more general case where finite exponential moments cannot be assumed. We also provide approximations of the tail bounds in terms of the more prevalent Orlicz norms or their `incomplete'' versions under suitable moment conditions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Deterministic Coreset Construction via Adaptive Sensitivity Trimming
We develop a rigorous framework for deterministic coreset construction in empirical risk minimization (ERM). Our central contribution is the Adaptive Deterministic Uniform-Weight Trimming (ADUWT) algorithm, which constru…
Instance-Dependent Generalization Bounds via Optimal Transport
Existing generalization bounds fail to explain crucial factors that drive the generalization of modern neural networks. Since such bounds often hold uniformly over all parameters, they suffer from over-parametrization an…
Generalization BoundsInductive BiasAn Exponential Efron-Stein Inequality for Lq Stable Learning Rules
There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there …
Generalization BoundsVariance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards
This paper presents two algorithms, AdaOFUL and VARA, for online sequential decision-making in the presence of heavy-tailed rewards with only finite variances. For linear stochastic bandits, we address the issue of heavy…
Decision Makingregressionreinforcement-learningReinforcement Learning (RL)+1Nearly Minimax Optimal Regret for Multinomial Logistic Bandit
In this paper, we study the contextual multinomial logit (MNL) bandit problem in which a learning agent sequentially selects an assortment based on contextual information, and user feedback follows an MNL choice model. T…