Top-$k$ Regularization for Supervised Feature Selection
Feature selection identifies subsets of informative features and reduces dimensions in the original feature space, helping provide insights into data generation or a variety of domain problems. Existing methods mainly depend on feature scoring functions or sparse regularizations; nonetheless, they have limited ability to reconcile the representativeness and inter-correlations of features. In this paper, we introduce a novel, simple yet effective regularization approach, named top-$k$ regularization, to supervised feature selection in regression and classification tasks. Structurally, the top-$k$ regularization induces a sub-architecture on the architecture of a learning model to boost its ability to select the most informative features and model complex nonlinear relationships simultaneously. Theoretically, we derive and mathematically prove a uniform approximation error bound for using this approach to approximate high-dimensional sparse functions. Extensive experiments on a wide variety of benchmarking datasets show that the top-$k$ regularization is effective and stable for supervised feature selection.
Code (0)
등록된 구현이 없습니다.
Tasks
Benchmarkingfeature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Joint Adaptive Graph and Structured Sparsity Regularization for Unsupervised Feature Selection
Feature selection is an important data preprocessing in data mining and machine learning which can be used to reduce the feature dimension without deteriorating model's performance. Since obtaining annotated data is labo…
feature selectionSparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection
In the field of data mining, how to deal with high-dimensional data is an inevitable problem. Unsupervised feature selection has attracted more and more attention because it does not rely on labels. The performance of sp…
feature selectionSimilarity Preserving Unsupervised Feature Selection based on Sparse Learning
Various feature selection methods have been recently proposed on different applications to reduce the computational burden of machine learning algorithms as well as the complexity of learned models. Preserving sample sim…
feature selectionSparse LearningUnsupervised feature selection via self-paced learning and low-redundant regularization
Much more attention has been paid to unsupervised feature selection nowadays due to the emergence of massive unlabeled data. The distribution of samples and the latent effect of training a learning method using samples i…
feature selectionSupervised Feature Selection in Graphs with Path Coding Penalties and Network Flows
We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network. In this context, it is of much interest to automatically select a subgraph with few connect…
feature selection