Faithful Variable Screening for High-Dimensional Convex Regression
We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the relevant variables. Our approach is a two-stage quadratic programming method that estimates a sum of one-dimensional convex functions, followed by one-dimensional concave regression fits on the residuals. In contrast to previous methods for sparse additive models, the optimization is finite dimensional and requires no tuning parameters for smoothness. Under appropriate assumptions, we prove that the procedure is faithful in the population setting, yielding no false negatives. We give a finite sample statistical analysis, and introduce algorithms for efficiently carrying out the required quadratic programs. The approach leads to computational and statistical advantages over fitting a full model, and provides an effective, practical approach to variable screening in convex regression.
Code (0)
등록된 구현이 없습니다.
Tasks
Additive modelsregressionVariable SelectionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Selective Factor Extraction in High Dimensions
This paper studies simultaneous feature selection and extraction in supervised and unsupervised learning. We propose and investigate selective reduced rank regression for constructing optimal explanatory factors from a p…
feature selectionModel SelectionVariable SelectionVocal Bursts Intensity PredictionScreening Rules for Lasso with Non-Convex Sparse Regularizers
Leveraging on the convexity of the Lasso problem , screening rules help in accelerating solvers by discarding irrelevant variables, during the optimization process. However, because they provide better theoretical guaran…
High-Dimensional Screening Using Multiple Grouping of Variables
Screening is the problem of finding a superset of the set of non-zero entries in an unknown p-dimensional vector \beta* given n noisy observations. Naturally, we want this superset to be as small as possible. We propose …
Variable SelectionVocal Bursts Intensity PredictionExSIS: Extended Sure Independence Screening for Ultrahigh-dimensional Linear Models
Statistical inference can be computationally prohibitive in ultrahigh-dimensional linear models. Correlation-based variable screening, in which one leverages marginal correlations for removal of irrelevant variables from…
On the consistency theory of high dimensional variable screening
Variable screening is a fast dimension reduction technique for assisting high dimensional feature selection. As a preselection method, it selects a moderate size subset of candidate variables for further refining via fea…
Computational EfficiencyDimensionality Reductionfeature selectionVocal Bursts Intensity Prediction