On the sample complexity of parameter estimation in logistic regression with normal design
The logistic regression model is one of the most popular data generation model in noisy binary classification problems. In this work, we study the sample complexity of estimating the parameters of the logistic regression model up to a given $\ell_2$ error, in terms of the dimension and the inverse temperature, with standard normal covariates. The inverse temperature controls the signal-to-noise ratio of the data generation process. While both generalization bounds and asymptotic performance of the maximum-likelihood estimator for logistic regression are well-studied, the non-asymptotic sample complexity that shows the dependence on error and the inverse temperature for parameter estimation is absent from previous analyses. We show that the sample complexity curve has two change-points in terms of the inverse temperature, clearly separating the low, moderate, and high temperature regimes.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationGeneralization Boundsparameter estimationregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning sparse generalized linear models with binary outcomes via iterative hard thresholding
In statistics, generalized linear models (GLMs) are widely used for modeling data and can expressively capture potential nonlinear dependence of the model's outcomes on its covariates. Within the broad family of GLMs, th…
Binary Classificationparameter estimationregressionMeta Learning for High-dimensional Ising Model Selection Using $\ell_1$-regularized Logistic Regression
In this paper, we consider the meta learning problem for estimating the graphs associated with high-dimensional Ising models, using the method of $\ell_1$-regularized logistic regression for neighborhood selection of eac…
Meta-LearningModel SelectionregressionExact Recovery of Sparse Binary Vectors from Generalized Linear Measurements
We consider the problem of exact recovery of a $k$-sparse binary vector from generalized linear measurements (such as logistic regression). We analyze the linear estimation algorithm (Plan, Vershynin, Yudovina, 2017), an…
2kQuantizationregressionA Unified Approach to Learning Ising Models: Beyond Independence and Bounded Width
We revisit the problem of efficiently learning the underlying parameters of Ising models from data. Current algorithmic approaches achieve essentially optimal sample complexity when given i.i.d. samples from the stationa…
regressionA Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression
This paper considers the problem of matrix-variate logistic regression. It derives the fundamental error threshold on estimating low-rank coefficient matrices in the logistic regression problem by obtaining a lower bound…
regression