Evaluation and Optimization of Leave-one-out Cross-validation for the Lasso
I develop an algorithm to produce the piecewise quadratic that computes leave-one-out cross-validation for the lasso as a function of its hyperparameter. The algorithm can be used to find exact hyperparameters that optimize leave-one-out cross-validation either globally or locally, and its practicality is demonstrated on real-world data sets. I also show how the algorithm can be modified to compute approximate leave-one-out cross-validation, making it suitable for larger data sets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Online Hyperparameter Search Interleaved with Proximal Parameter Updates
There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or…
Hyperparameter OptimizationAn Homotopy Algorithm for the Lasso with Online Observations
It has been shown that the problem of $\ell_1$-penalized least-square regression commonly referred to as the Lasso or Basis Pursuit DeNoising leads to solutions that are sparse and therefore achieves model selection. We …
compressed sensingDenoisingModel SelectionregressionTheoretical Analysis of Leave-one-out Cross Validation for Non-differentiable Penalties under High-dimensional Settings
Despite a large and significant body of recent work focused on estimating the out-of-sample risk of regularized models in the high dimensional regime, a theoretical understanding of this problem for non-differentiable pe…
Distributional bias compromises leave-one-out cross-validation
Cross-validation is a common method for estimating the predictive performance of machine learning models. In a data-scarce regime, where one typically wishes to maximize the number of instances used for training the mode…
Hyperparameter OptimizationRobust sketching for multiple square-root LASSO problems
Many learning tasks, such as cross-validation, parameter search, or leave-one-out analysis, involve multiple instances of similar problems, each instance sharing a large part of learning data with the others. We introduc…