Sharp Convergence Rates for Forward Regression in High-Dimensional Sparse Linear Models
Forward regression is a statistical model selection and estimation procedure which inductively selects covariates that add predictive power into a working statistical regression model. Once a model is selected, unknown regression parameters are estimated by least squares. This paper analyzes forward regression in high-dimensional sparse linear models. Probabilistic bounds for prediction error norm and number of selected covariates are proved. The analysis in this paper gives sharp rates and does not require beta-min or irrepresentability conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Model SelectionregressionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Large-Step Training Dynamics of a Two-Factor Linear Transformer Model
Gradient-flow analyses show that simplified linear transformers can learn the in-context linear-regression algorithm, but they do not explain the finite-step behavior of gradient descent at large learning rates. Motivate…
Sharp global convergence guarantees for iterative nonconvex optimization: A Gaussian process perspective
We consider a general class of regression models with normally distributed covariates, and the associated nonconvex problem of fitting these models from data. We develop a general recipe for analyzing the convergence of …
parameter estimationRetrievalEquivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals
We study the posterior contraction rates of a Bayesian method with Gaussian process priors in nonparametric regression and its plug-in property for differential operators. For a general class of kernels, we establish con…
Gaussian ProcessesregressionImproving the Predictive Performances of $k$ Nearest Neighbors Learning by Efficient Variable Selection
This paper computationally demonstrates a sharp improvement in predictive performance for $k$ nearest neighbors thanks to an efficient forward selection of the predictor variables. We show both simulated and real-world d…
regressionVariable SelectionSharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes
Fine-tuning vision language models (VLMs) has achieved remarkable performance across various downstream tasks; yet, it requires access to model gradients through backpropagation (BP), making them unsuitable for memory-co…
Language ModelingLanguage Modelling