On model selection consistency of regularized M-estimators
Regularized M-estimators are used in diverse areas of science and engineering to fit high-dimensional models with some low-dimensional structure. Usually the low-dimensional structure is encoded by the presence of the (unknown) parameters in some low-dimensional model subspace. In such settings, it is desirable for estimates of the model parameters to be \emph{model selection consistent}: the estimates also fall in the model subspace. We develop a general framework for establishing consistency and model selection consistency of regularized M-estimators and show how it applies to some special cases of interest in statistical learning. Our analysis identifies two key properties of regularized M-estimators, referred to as geometric decomposability and irrepresentability, that ensure the estimators are consistent and model selection consistent.
Code (0)
등록된 구현이 없습니다.
Tasks
modelModel SelectionSimilar Papers 제목 키워드 기반
Generalized Information Criteria for Structured Sparse Models
Regularized m-estimators are widely used due to their ability of recovering a low-dimensional model in high-dimensional scenarios. Some recent efforts on this subject focused on creating a unified framework for establish…
Model SelectionregressionStatistical Estimation of High-Dimensional Vector Autoregressive Models
High-dimensional vector autoregressive (VAR) models are important tools for the analysis of multivariate time series. This paper focuses on high-dimensional time series and on the different regularized estimation procedu…
Time SeriesTime Series AnalysisVocal Bursts Intensity PredictionThe Adaptive $τ$-Lasso: Robustness and Oracle Properties
This paper introduces a new regularized version of the robust $\tau$-regression estimator for analyzing high-dimensional datasets subject to gross contamination in the response variables and covariates. The resulting est…
regressionVariable SelectionRobust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression
Heavy-tailed error distributions and predictors with anomalous values are ubiquitous in high-dimensional regression problems and can seriously jeopardize the validity of statistical analyses if not properly addressed. Fo…
regressionVariable SelectionClosed-form Estimators for High-dimensional Generalized Linear Models
We propose a class of closed-form estimators for GLMs under high-dimensional sampling regimes. Our class of estimators is based on deriving closed-form variants of the vanilla unregularized MLE but which are (a) well-def…
FormVariable SelectionVocal Bursts Intensity Prediction