Revisiting Random Channel Pruning for Neural Network Compression
Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of neural networks. There has been a flurry of algorithms that try to solve this practical problem, each being claimed effective in some ways. Yet, a benchmark to compare those algorithms directly is lacking, mainly due to the complexity of the algorithms and some custom settings such as the particular network configuration or training procedure. A fair benchmark is important for the further development of channel pruning. Meanwhile, recent investigations reveal that the channel configurations discovered by pruning algorithms are at least as important as the pre-trained weights. This gives channel pruning a new role, namely searching the optimal channel configuration. In this paper, we try to determine the channel configuration of the pruned models by random search. The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning. We show that this simple strategy works quite well compared with other channel pruning methods. We also show that under this setting, there are surprisingly no clear winners among different channel importance evaluation methods, which then may tilt the research efforts into advanced channel configuration searching methods.
Code (1)
Tasks
Neural Network CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Conditional Automated Channel Pruning for Deep Neural Networks
Model compression aims to reduce the redundancy of deep networks to obtain compact models. Recently, channel pruning has become one of the predominant compression methods to deploy deep models on resource-constrained dev…
Model CompressionJoint Channel and Weight Pruning for Model Acceleration on Moblie Devices
For practical deep neural network design on mobile devices, it is essential to consider the constraints incurred by the computational resources and the inference latency in various applications. Among deep network accele…
Model CompressionPCAS: Pruning Channels with Attention Statistics for Deep Network Compression
Compression techniques for deep neural networks are important for implementing them on small embedded devices. In particular, channel-pruning is a useful technique for realizing compact networks. However, many convention…
channel selectionGradual Channel Pruning while Training using Feature Relevance Scores for Convolutional Neural Networks
The enormous inference cost of deep neural networks can be scaled down by network compression. Pruning is one of the predominant approaches used for deep network compression. However, existing pruning techniques have one…
Model CompressionC2S2: Cost-aware Channel Sparse Selection for Progressive Network Pruning
This paper describes a channel-selection approach for simplifying deep neural networks. Specifically, we propose a new type of generic network layer, called pruning layer, to seamlessly augment a given pre-trained model …
channel selectionimage-classificationImage ClassificationNetwork Pruning+1