Network Pruning using Adaptive Exemplar Filters
Popular network pruning algorithms reduce redundant information by optimizing hand-crafted models, and may cause suboptimal performance and long time in selecting filters. We innovatively introduce adaptive exemplar filters to simplify the algorithm design, resulting in an automatic and efficient pruning approach called EPruner. Inspired by the face recognition community, we use a message passing algorithm Affinity Propagation on the weight matrices to obtain an adaptive number of exemplars, which then act as the preserved filters. EPruner breaks the dependency on the training data in determining the "important" filters and allows the CPU implementation in seconds, an order of magnitude faster than GPU based SOTAs. Moreover, we show that the weights of exemplars provide a better initialization for the fine-tuning. On VGGNet-16, EPruner achieves a 76.34%-FLOPs reduction by removing 88.80% parameters, with 0.06% accuracy improvement on CIFAR-10. In ResNet-152, EPruner achieves a 65.12%-FLOPs reduction by removing 64.18% parameters, with only 0.71% top-5 accuracy loss on ILSVRC-2012. Our code can be available at https://github.com/lmbxmu/EPruner.
Code (1)
Tasks
CPUFace RecognitionGPUNetwork PruningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
Pruning redundant filters in CNN models has received growing attention. In this paper, we propose an adaptive binary search-first hybrid pyramid- and clustering-based (ABSHPC-based) method for pruning filters automatical…
ClusteringProtective Self-Adaptive Pruning to Better Compress DNNs
Adaptive network pruning approach has recently drawn significant attention due to its excellent capability to identify the importance and redundancy of layers and filters and customize a suitable pruning solution. Howeve…
Network PruningSlimming Neural Networks using Adaptive Connectivity Scores
In general, deep neural network (DNN) pruning methods fall into two categories: 1) Weight-based deterministic constraints, and 2) Probabilistic frameworks. While each approach has its merits and limitations there are a s…
SensitivityInterspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNs
Unstructured pruning is well suited to reduce the memory footprint of convolutional neural networks (CNNs), both at training and inference time. CNNs contain parameters arranged in $K \times K$ filters. Standard unstruct…
Filter Pruning by Switching to Neighboring CNNs with Good Attributes
Filter pruning is effective to reduce the computational costs of neural networks. Existing methods show that updating the previous pruned filter would enable large model capacity and achieve better performance. However, …
Attributeimage-classificationImage Classification