On the Differences between L2-Boosting and the Lasso
We prove that L2-Boosting lacks a theoretical property which is central to the behaviour of l1-penalized methods such as basis pursuit and the Lasso: Whereas l1-penalized methods are guaranteed to recover the sparse parameter vector in a high-dimensional linear model under an appropriate restricted nullspace property, L2-Boosting is not guaranteed to do so. Hence, L2-Boosting behaves quite differently from l1-penalized methods when it comes to parameter recovery/estimation in high-dimensional linear models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Sparse-group boosting -- Unbiased group and variable selection
In the presence of grouped covariates, we propose a framework for boosting that allows to enforce sparsity within and between groups. By using component-wise and group-wise gradient boosting at the same time with adjuste…
Variable SelectionLassoed Boosting and Linear Prediction in the Equities Market
We consider a two-stage estimation method for linear regression. First, it uses the lasso in Tibshirani (1996) to screen variables and, second, re-estimates the coefficients using the least-squares boosting method in Fri…
PredictionregressionHigh-Dimensional $L_2$Boosting: Rate of Convergence
Boosting is one of the most significant developments in machine learning. This paper studies the rate of convergence of $L_2$Boosting, which is tailored for regression, in a high-dimensional setting. Moreover, we introdu…
Vocal Bursts Intensity PredictionA New Perspective on Boosting in Linear Regression via Subgradient Optimization and Relatives
In this paper we analyze boosting algorithms in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely t…
regressionLassoed Tree Boosting
Gradient boosting performs exceptionally in most prediction problems and scales well to large datasets. In this paper we prove that a ``lassoed'' gradient boosted tree algorithm with early stopping achieves faster than $…
regression