Feature Selection via Maximizing Distances between Class Conditional Distributions
For many data-intensive tasks, feature selection is an important preprocessing step. However, most existing methods do not directly and intuitively explore the intrinsic discriminative information of features. We propose a novel feature selection framework based on the distance between class conditional distributions, measured by integral probability metrics (IPMs). Our framework directly explores the discriminative information of features in the sense of distributions for supervised classification. We analyze the theoretical and practical aspects of IPMs for feature selection, construct criteria based on IPMs. We propose several variant feature selection methods of our framework based on the 1-Wasserstein distance and implement them on real datasets from different domains. Experimental results show that our framework can outperform state-of-the-art methods in terms of classification accuracy and robustness to perturbations.
Code (0)
등록된 구현이 없습니다.
Tasks
feature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Supervised Quadratic Feature Analysis: Information Geometry Approach for Dimensionality Reduction
Supervised dimensionality reduction aims to map labeled data to a low-dimensional feature space while maximizing class discriminability. Directly computing discriminability is often impractical, so an alternative approac…
Dimensionality ReductionMetric LearningSupervised dimensionality reductionImproving Adversarial Robustness by Encouraging Discriminative Features
Deep neural networks (DNNs) have achieved state-of-the-art results in various pattern recognition tasks. However, they perform poorly on out-of-distribution adversarial examples i.e. inputs that are specifically crafted …
Adversarial RobustnessDeep Simplex Classifier for Maximizing the Margin in Both Euclidean and Angular Spaces
The classification loss functions used in deep neural network classifiers can be grouped into two categories based on maximizing the margin in either Euclidean or angular spaces. Euclidean distances between sample vector…
ClassificationOpen Set LearningA unifying mutual information view of metric learning: cross-entropy vs. pairwise losses
Recently, substantial research efforts in Deep Metric Learning (DML) focused on designing complex pairwise-distance losses, which require convoluted schemes to ease optimization, such as sample mining or pair weighting. …
Metric LearningS$^2$FS: Spatially-Aware Separability-Driven Feature Selection in Fuzzy Decision Systems
Feature selection is crucial for fuzzy decision systems (FDSs), as it identifies informative features and eliminates rule redundancy, thereby enhancing predictive performance and interpretability. Most existing methods e…