Feature subset selection for kernel SVM classification via mixed-integer optimization

2022-05-28 · Ryuta Tamura, Yuichi Takano, Ryuhei Miyashiro

We study the mixed-integer optimization (MIO) approach to feature subset selection in nonlinear kernel support vector machines (SVMs) for binary classification. First proposed for linear regression in the 1970s, this approach has recently moved into the spotlight with advances in optimization algorithms and computer hardware. The goal of this paper is to establish an MIO approach for selecting the best subset of features for kernel SVM classification. To measure the performance of subset selection, we use the kernel-target alignment, which is the distance between the centroids of two response classes in a high-dimensional feature space. We propose a mixed-integer linear optimization (MILO) formulation based on the kernel-target alignment for feature subset selection, and this MILO problem can be solved to optimality using optimization software. We also derive a reduced version of the MILO problem to accelerate our MILO computations. Experimental results show good computational efficiency for our MILO formulation with the reduced problem. Moreover, our method can often outperform the linear-SVM-based MILO formulation and recursive feature elimination in prediction performance, especially when there are relatively few data instances.

📄 PDF Abstract BibTeX arXiv:2205.14325

Code (0)

등록된 구현이 없습니다.

Tasks

Binary Classification Computational Efficiency

Methods 이 논문이 사용한 방법론

SVM A Support Vector Machine, or SVM, is a non-parametric supervised learning model. For non-linear classification and regression, they utilise the kernel trick to map inputs…

Linear Regression Linear Regression is a method for modelling a relationship between a dependent variable and independent variables. These models can be fit with numerous approaches. The most…

Generalized Fisher Score for Feature Selection

2012-02-14 · Quanquan Gu, Zhenhui Li, Jiawei Han

Fisher score is one of the most widely used supervised feature selection methods. However, it selects each feature independently according to their scores under the Fisher criterion, which leads to a suboptimal subset of…

feature selection

Latent regularization for feature selection using kernel methods in tumor classification

2020-04-10 · Martin Palazzo, Patricio Yankilevich, Pierre Beauseroy

The transcriptomics of cancer tumors are characterized with tens of thousands of gene expression features. Patient prognosis or tumor stage can be assessed by machine learning techniques like supervised classification ta…

Classification Dimensionality Reduction feature selection General Classification+1

Large-Margin Multiple Kernel Learning for Discriminative Features Selection and Representation Learning

2019-03-08 · Babak Hosseini, Barbara Hammer

Multiple kernel learning (MKL) algorithms combine different base kernels to obtain a more efficient representation in the feature space. Focusing on discriminative tasks, MKL has been used successfully for feature select…

Binary Classification feature selection General Classification Representation Learning

Optimal Sparse Kernel Learning for Hyperspectral Anomaly Detection

2015-06-08 · Zhimin Peng, Prudhvi Gurram, Heesung Kwon, Wotao Yin

In this paper, a novel framework of sparse kernel learning for Support Vector Data Description (SVDD) based anomaly detection is presented. In this work, optimal sparse feature selection for anomaly detection is first mo…

Anomaly Detection feature selection

A Supervised Feature Selection Method For Mixed-Type Data using Density-based Feature Clustering

2021-11-10 · Xuyang Yan, Mrinmoy Sarkar, Biniam Gebru, Shabnam Nazmi 외

Feature selection methods are widely used to address the high computational overheads and curse of dimensionality in classifying high-dimensional data. Most conventional feature selection methods focus on handling homoge…

Clustering feature selection