Feature importance scores and lossless feature pruning using Banzhaf power indices
Understanding the influence of features in machine learning is crucial to interpreting models and selecting the best features for classification. In this work we propose the use of principles from coalitional game theory to reason about importance of features. In particular, we propose the use of the Banzhaf power index as a measure of influence of features on the outcome of a classifier. We show that features having Banzhaf power index of zero can be losslessly pruned without damage to classifier accuracy. Computing the power indices does not require having access to data samples. However, if samples are available, the indices can be empirically estimated. We compute Banzhaf power indices for a neural network classifier on real-life data, and compare the results with gradient-based feature saliency, and coefficients of a logistic regression model with $L_1$ regularization.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature ImportanceGeneral ClassificationSimilar Papers 제목 키워드 기반
Adaptive MLP Pruning for Large Vision Transformers
Large vision transformers present impressive scalability, as their performance can be well improved with increased model capacity. Nevertheless, their cumbersome parameters results in exorbitant computational and memory …
Probabilistic Connection Importance Inference and Lossless Compression of Deep Neural Networks
Deep neural networks (DNNs) can be huge in size, requiring a considerable a mount of energy and computational resources to operate, which limits their applications in numerous scenarios. It is thus of interest to compres…
Frustratingly Easy Task-aware Pruning for Large Language Models
Pruning provides a practical solution to reduce the resources required to run large language models (LLMs) to benefit from their effective capabilities as well as control their cost for training and inference. Research o…
MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs
For multimodal large language models (MLLMs), visual information is relatively sparse compared with text. As a result, research on visual pruning emerges for efficient inference. Current approaches typically measure toke…
Exploiting Adaptive Channel Pruning for Communication-Efficient Split Learning
Split learning (SL) transfers most of the training workload to the server, which alleviates computational burden on client devices. However, the transmission of intermediate feature representations, referred to as smashe…