High Dimensional Classification through $\ell_0$-Penalized Empirical Risk Minimization
We consider a high dimensional binary classification problem and construct a classification procedure by minimizing the empirical misclassification risk with a penalty on the number of selected features. We derive non-asymptotic probability bounds on the estimated sparsity as well as on the excess misclassification risk. In particular, we show that our method yields a sparse solution whose l0-norm can be arbitrarily close to true sparsity with high probability and obtain the rates of convergence for the excess misclassification risk. The proposed procedure is implemented via the method of mixed integer linear programming. Its numerical performance is illustrated in Monte Carlo experiments.
Code (1)
Tasks
Binary ClassificationClassificationGeneral ClassificationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification
Neural networks are usually not the tool of choice for nonparametric high-dimensional problems where the number of input features is much larger than the number of observations. Though neural networks can approximate com…
General ClassificationregressionVocal Bursts Intensity PredictionFinite-sample and asymptotic analysis of generalization ability with an application to penalized regression
In this paper, we study the performance of extremum estimators from the perspective of generalization ability (GA): the ability of a model to predict outcomes in new samples from the same population. By adapting the clas…
regressionAlternating direction method of multipliers for penalized zero-variance discriminant analysis
We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuris…
feature selectionGeneral ClassificationTime SeriesTime Series Analysis+1Sparse Distance Weighted Discrimination
Distance weighted discrimination (DWD) was originally proposed to handle the data piling issue in the support vector machine. In this paper, we consider the sparse penalized DWD for high-dimensional classification. The s…
Computational EfficiencyGeneral ClassificationMulticlass classification by sparse multinomial logistic regression
In this paper we consider high-dimensional multiclass classification by sparse multinomial logistic regression. We propose first a feature selection procedure based on penalized maximum likelihood with a complexity penal…
Classificationfeature selectionGeneral Classificationregression