paper-with-me

홈 › Papers

Early stopping and polynomial smoothing in regression with reproducing kernels

2020-07-14 · Yaroslav Averyanov, Alain Celisse

In this paper, we study the problem of early stopping for iterative learning algorithms in a reproducing kernel Hilbert space (RKHS) in the nonparametric regression framework. In particular, we work with the gradient descent and (iterative) kernel ridge regression algorithms. We present a data-driven rule to perform early stopping without a validation set that is based on the so-called minimum discrepancy principle. This method enjoys only one assumption on the regression function: it belongs to a reproducing kernel Hilbert space (RKHS). The proposed rule is proved to be minimax-optimal over different types of kernel spaces, including finite-rank and Sobolev smoothness classes. The proof is derived from the fixed-point analysis of the localized Rademacher complexities, which is a standard technique for obtaining optimal rates in the nonparametric regression literature. In addition to that, we present simulation results on artificial datasets that show the comparable performance of the designed rule with respect to other stopping rules such as the one determined by V-fold cross-validation.

📄 PDF Abstract BibTeX arXiv:2007.06827

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Methods 이 논문이 사용한 방법론

Early Stopping Early Stopping is a regularization technique for deep neural networks that stops training when parameter updates no longer begin to yield improves on a validation set. In…

Similar Papers 제목 키워드 기반

Early stopping and non-parametric regression: An optimal data-dependent stopping rule

2013-06-15 · Garvesh Raskutti, Martin J. Wainwright, Bin Yu

The strategy of early stopping is a regularization technique based on choosing a stopping time for an iterative algorithm. Focusing on non-parametric regression in a reproducing kernel Hilbert space, we analyze the early…

regression

Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression

2025-02-18 · Jingfeng Wu, Peter Bartlett, Matus Telgarsky, Bin Yu

In overparameterized logistic regression, gradient descent (GD) iterates diverge in norm while converging in direction to the maximum $\ell_2$-margin solution -- a phenomenon known as the implicit bias of GD. This work i…

regression

Gradient Descent Finds Over-Parameterized Neural Networks with Sharp Generalization for Nonparametric Regression

2024-11-05 · Yingzhen Yang, Ping Li

We study nonparametric regression by an over-parameterized two-layer neural network trained by gradient descent (GD) in this paper. We show that, if the neural network is trained by GD with early stopping, then the train…

regression

Surrogate-based Real-time Curbside Management for Ride-hailing and Delivery Operations

2023-10-05 · Suyash C. Vishnoi, Michele D. Simoni

The present work investigates surrogate model-based optimization for real-time curbside traffic management operations. An optimization problem is formulated to minimize the congestion on roadway segments caused by vehicl…

ManagementModel Predictive Controlregression

Generalization of Overparametrized Deep Neural Network Under Noisy Observations

2021-09-29 · ICLR 2022 4 · Namjoon Suh, Hyunouk Ko, Xiaoming Huo

We study the generalization properties of the overparameterized deep neural network (DNN) with Rectified Linear Unit (ReLU) activations. Under the non-parametric regression framework, it is assumed that the ground-truth …