High-dimensional classification by sparse logistic regression

2017-06-26 · Felix Abramovich, Vadim Grinshtein

We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for the resulting misclassification excess risk. The bounds can be reduced under the additional low-noise condition. The proposed complexity penalty is remarkably related to the VC-dimension of a set of sparse linear classifiers. Implementation of any complexity penalty-based criterion, however, requires a combinatorial search over all possible models. To find a model selection procedure computationally feasible for high-dimensional data, we extend the Slope estimator for logistic regression and show that under an additional weighted restricted eigenvalue condition it is rate-optimal in the minimax sense.

📄 PDF Abstract BibTeX arXiv:1706.08344

Code (1)

stat-lu/PhDpos

Tasks

Binary Classification Classificationfeature selectionGeneral ClassificationModel Selectionregression Vocal Bursts Intensity Prediction

Methods 이 논문이 사용한 방법론

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

A Safe Screening Rule for Sparse Logistic Regression

2013-07-16 · NeurIPS 2014 12 · Jie Wang, Jiayu Zhou, Jun Liu, Peter Wonka 외

The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implem…

feature selection regression

Information-theoretical label embeddings for large-scale image classification

2016-07-19 · François Chollet

We present a method for training multi-label, massively multi-class image classification models, that is faster and more accurate than supervision via a sigmoid cross-entropy loss (logistic regression). Our method consis…

Classification General Classification image-classification Image Classification+1

Multiclass classification by sparse multinomial logistic regression

2020-03-04 · Felix Abramovich, Vadim Grinshtein, Tomer Levy

In this paper we consider high-dimensional multiclass classification by sparse multinomial logistic regression. We propose first a feature selection procedure based on penalized maximum likelihood with a complexity penal…

Classification feature selection General Classification regression

Extreme Sparse Multinomial Logistic Regression: A Fast and Robust Framework for Hyperspectral Image Classification

2017-09-08 · Faxian Cao, Zhijing Yang, Jinchang Ren, Wing-Kuen Ling

Although the sparse multinomial logistic regression (SMLR) has provided a useful tool for sparse classification, it suffers from inefficacy in dealing with high dimensional features and manually set initial regressor val…

Attribute Classification General Classification Hyperspectral Image Classification+3

PIANO: A Fast Parallel Iterative Algorithm for Multinomial and Sparse Multinomial Logistic Regression

2020-02-21 · R. Jyothi, P. Babu

Multinomial Logistic Regression is a well-studied tool for classification and has been widely used in fields like image processing, computer vision and, bioinformatics, to name a few. Under a supervised classification sc…