A Survey of Tuning Parameter Selection for High-dimensional Regression
Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which determines the amount of regularization and hence the sparsity level of the fitted model. The optimal choice of tuning parameter depends on both the structure of the design matrix and the unknown random error distribution (variance, tail behavior, etc). This article reviews the current literature of tuning parameter selection for high-dimensional regression from both theoretical and practical perspectives. We discuss various strategies that choose the tuning parameter to achieve prediction accuracy or support recovery. We also review several recently proposed methods for tuning-free high-dimensional regression.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSurveyVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
A Survey and Comparative Evaluation of Intrinsic Dimension Estimators under the Manifold Hypothesis
The manifold hypothesis suggests that high-dimensional data often lie on or near a low-dimensional manifold. Estimating the dimension of this manifold is essential for leveraging its structure, yet existing work on dimen…
Don't Fall for Tuning Parameters: Tuning-Free Variable Selection in High Dimensions With the TREX
Lasso is a seminal contribution to high-dimensional statistics, but it hinges on a tuning parameter that is difficult to calibrate in practice. A partial remedy for this problem is Square-Root Lasso, because it inherentl…
Computational EfficiencyVariable SelectionFeature Selection Strategies for Classifying High Dimensional Astronomical Data Sets
The amount of collected data in many scientific fields is increasing, all of them requiring a common task: extract knowledge from massive, multi parametric data sets, as rapidly and efficiently possible. This is especial…
Astronomyfeature selectionGeneral ClassificationVocal Bursts Intensity PredictionOn the Selection of Tuning Parameters for Patch-Stitching Embedding Methods
While classical scaling, just like principal component analysis, is parameter-free, other methods for embedding multivariate data require the selection of one or several tuning parameters. This tuning can be difficult du…
Dimensionality ReductionPath Thresholding: Asymptotically Tuning-Free High-Dimensional Sparse Regression
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that tra…
Model SelectionregressionVocal Bursts Intensity Prediction