Reliable Identification of Redundant Kernels for Convolutional Neural Network Compression
To compress deep convolutional neural networks (CNNs) with large memory footprint and long inference time, this paper proposes a novel pruning criterion using layer-wised Ln-norm of feature maps. Different from existing pruning criteria, which are mainly based on L1-norm of convolution kernels, the proposed method utilizes Ln-norm of output feature maps after non-linear activations, where n is a variable, increasing from 1 at the first convolution layer to inf at the last convolution layer. With the ability of accurately identifying unimportant convolution kernels, the proposed method achieves a good balance between model size and inference accuracy. The experiments on ImageNet and the successful application in railway surveillance system show that the proposed method outperforms existing kernel-norm-based methods and is generally applicable to any deep neural network with convolution operations.
Code (0)
등록된 구현이 없습니다.
Tasks
Neural Network CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Clustering Convolutional Kernels to Compress Deep Neural Networks
In this paper, we propose a novel method to compress CNNs by reconstructing the network from a small set of spatial convolution kernels. Starting from a pre-trained model, we extract representative 2D kernel centroids us…
ClusteringGeneral ClassificationNetwork PruningRecurrent Convolutions: A Model Compression Point of View
Recurrent convolution (RC) shares the same convolutional kernels and unrolls them multiple times, which is originally proposed to model time-space signals. We suggest that RC can be viewed as a model compression strategy…
Model CompressionRolling Shutter CorrectionCompressing 3DCNNs Based on Tensor Train Decomposition
Three dimensional convolutional neural networks (3DCNNs) have been applied in many tasks, e.g., video and 3D point cloud recognition. However, due to the higher dimension of convolutional kernels, the space complexity of…
Hand Gesture RecognitionHand-Gesture RecognitionNeural Network CompressionQuantization+1Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning
The emergence of deep and large-scale spiking neural networks (SNNs) exhibiting high performance across diverse complex datasets has led to a need for compressing network models due to the presence of a significant numbe…
Model CompressionNetwork PruningSparse LearningDiscrimination-aware Network Pruning for Deep Model Compression
We study network pruning which aims to remove redundant channels/kernels and hence speed up the inference of deep networks. Existing pruning methods either train from scratch with sparsity constraints or minimize the rec…
Face Recognitionimage-classificationImage Classificationmodel+2