Tuning parameter calibration for $\ell_1$-regularized logistic regression
Feature selection is a standard approach to understanding and modeling high-dimensional classification data, but the corresponding statistical methods hinge on tuning parameters that are difficult to calibrate. In particular, existing calibration schemes in the logistic regression framework lack any finite sample guarantees. In this paper, we introduce a novel calibration scheme for $\ell_1$-penalized logistic regression. It is based on simple tests along the tuning parameter path and is equipped with optimal guarantees for feature selection. It is also amenable to easy and efficient implementations, and it rivals or outmatches existing methods in simulations and real data applications.
Code (0)
등록된 구현이 없습니다.
Tasks
feature selectionGeneral ClassificationregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bregman Distance to L1 Regularized Logistic Regression
In this work we investigate the relationship between Bregman distances and regularized Logistic Regression model. We present a detailed study of Bregman Distance minimization, a family of generalized entropy measures ass…
regressionNew Bounds for Hyperparameter Tuning of Regression Problems Across Instances
The task of tuning regularization coefficients in regularized regression models with provable guarantees across problem instances still poses a significant challenge in the literature. This paper investigates the sample …
$L_2$-Regularized Empirical Risk Minimization Guarantees Small Smooth Calibration Error
Calibration of predicted probabilities is critical for reliable machine learning, yet it is poorly understood how standard training procedures yield well-calibrated models. This work provides the first theoretical proof …
Improving Geopolitical Forecasts with Bayesian Networks
This study explores how Bayesian networks (BNs) can improve forecast accuracy compared to logistic regression and recalibration and aggregation methods, using data from the Good Judgment Project. Regularized logistic reg…
The Impact of Regularization on High-dimensional Logistic Regression
Logistic regression is commonly used for modeling dichotomous outcomes. In the classical setting, where the number of observations is much larger than the number of parameters, properties of the maximum likelihood estima…
regressionVocal Bursts Intensity Prediction