Binary Stochastic Filtering: a Method for Neural Network Size Minimization and Supervised Feature Selection
Binary Stochastic Filtering (BSF), the algorithm for feature selection and neuron pruning is proposed in this work. The method defines filtering layer which penalizes amount of the information involved in the training process. This information could be the input data or output of the previous layer, which directly leads to the feature selection or neuron pruning respectively, producing \textit{ad hoc} subset of features or selecting optimal number of neurons in each layer. Filtering layer stochastically passes or drops features based on individual weights, which are tuned with standard backpropagation algorithm during the training process. Multifold decrease of neural network size has been achieved in the experiments. Besides, the method was able to select minimal number of features, surpassing literature references by the accuracy/dimensionality ratio.
Code (1)
Tasks
feature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Particle Filtering Methods for Stochastic Optimization with Application to Large-Scale Empirical Risk Minimization
There is a recent interest in developing statistical filtering methods for stochastic optimization (FSO) by leveraging a probabilistic perspective of incremental proximity methods (IPMs). The existent FSO methods are der…
Binary ClassificationSchedulingStochastic OptimizationA Stochastic Optimization Framework for Fair Risk Minimization
Despite the success of large-scale empirical risk minimization (ERM) at achieving high accuracy across a variety of machine learning tasks, fair ERM is hindered by the incompatibility of fairness constraints with stochas…
Binary ClassificationFairnessStochastic OptimizationA variable metric proximal stochastic gradient method: an application to classification problems
Due to the continued success of machine learning and deep learning in particular, supervised classification problems are ubiquitous in numerous scientific fields. Training these models typically involves the minimization…
Binary ClassificationClassificationimage-classificationImage ClassificationThe Double Descent Behavior in Two Layer Neural Network for Binary Classification
Recent studies observed a surprising concept on model test error called the double descent phenomenon, where the increasing model complexity decreases the test error first and then the error increases and decreases again…
Binary ClassificationCo-clustering separately exchangeable network data
This article establishes the performance of stochastic blockmodels in addressing the co-clustering problem of partitioning a binary array into subsets, assuming only that the data are generated by a nonparametric process…
Clustering