Binary Linear Classification and Feature Selection via Generalized Approximate Message Passing

2014-01-05 · Justin Ziniel, Philip Schniter, Per Sederberg

For the problem of binary linear classification and feature selection, we propose algorithmic approaches to classifier design based on the generalized approximate message passing (GAMP) algorithm, recently proposed in the context of compressive sensing. We are particularly motivated by problems where the number of features greatly exceeds the number of training examples, but where only a few features suffice for accurate classification. We show that sum-product GAMP can be used to (approximately) minimize the classification error rate and max-sum GAMP can be used to minimize a wide variety of regularized loss functions. Furthermore, we describe an expectation-maximization (EM)-based scheme to learn the associated model parameters online, as an alternative to cross-validation, and we show that GAMP's state-evolution framework can be used to accurately predict the misclassification rate. Finally, we present a detailed numerical study to confirm the accuracy, speed, and flexibility afforded by our GAMP-based approaches to binary linear classification and feature selection.

📄 PDF Abstract BibTeX arXiv:1401.0872

Code (0)

등록된 구현이 없습니다.

Tasks

Classification Compressive Sensingfeature selectionGeneral Classification

Towards Searching Efficient and Accurate Neural Network Architectures in Binary Classification Problems

2021-01-16 · Yigit Alparslan, Ethan Jacob Moyer, Isamu Mclean Isozaki, Daniel Schwartz 외

In recent years, deep neural networks have had great success in machine learning and pattern recognition. Architecture size for a neural network contributes significantly to the success of any neural network. In this stu…

BIG-bench Machine Learning Binary Classification Classification General Classification

RENT -- Repeated Elastic Net Technique for Feature Selection

2020-09-27 · Anna Jenul, Stefan Schrunner, Kristian Hovde Liland, Ulf Geir Indahl 외

Feature selection is an essential step in data science pipelines to reduce the complexity associated with large datasets. While much research on this topic focuses on optimizing predictive performance, few studies invest…

Binary Classification feature selection

Optimizing Sparse Generalized Singular Vectors for Feature Selection in Proximal Support Vector Machines with Application to Breast and Ovarian Cancer Detection

2024-10-04 · Ugochukwu O. Ugwu, Michael Kirby

This paper presents approaches to compute sparse solutions of Generalized Singular Value Problem (GSVP). The GSVP is regularized by $\ell_1$-norm and $\ell_q$-penalty for $0<q<1$, resulting in the $\ell_1$-GSVP and $\ell…

Binary Classification feature selection

Consistent Binary Classification with Generalized Performance Metrics

2014-12-01 · NeurIPS 2014 12 · Oluwasanmi O. Koyejo, Nagarajan Natarajan, Pradeep K. Ravikumar, Inderjit S. Dhillon

Performance metrics for binary classification are designed to capture tradeoffs between four fundamental population quantities: true positives, false positives, true negatives and false negatives. Despite significant int…

Binary Classification Classification General Classification

Selection consistency of Lasso-based procedures for misspecified high-dimensional binary model and random regressors

2019-06-10 · Mariusz Kubkowski, Jan Mielniczuk

We consider selection of random predictors for high-dimensional regression problem with binary response for a general loss function. Important special case is when the binary model is semiparametric and the response func…

regression