CWP: Instance complexity weighted channel-wise soft masks for network pruning

2022-09-08 · Jiapeng Wang, Ming Ma, Zhenhua Yu

Existing differentiable channel pruning methods often attach scaling factors or masks behind channels to prune filters with less importance, and implicitly assume uniform contribution of input samples to filter importance. Specifically, the effects of instance complexity on pruning performance are not yet fully investigated in static network pruning. In this paper, we propose a simple yet effective differentiable network pruning method CWP based on instance complexity weighted filter importance scores. We define instance complexity related weight for each instance by giving higher weights to hard instances, and measure the weighted sum of instance-specific soft masks to model non-uniform contribution of different inputs, which encourages hard instances to dominate the pruning process and the model performance to be well preserved. In addition, we introduce a regularizer to maximize polarization of the masks, such that a sweet spot can be easily found to identify the filters to be pruned. Performance evaluations on various network architectures and datasets demonstrate CWP has advantages over the state-of-the-arts in pruning large networks. For instance, CWP improves the accuracy of ResNet56 on CIFAR-10 dataset by 0.32% aftering removing 64.11% FLOPs, and prunes 87.75% FLOPs of ResNet50 on ImageNet dataset with only 0.93% Top-1 accuracy loss.

📄 PDF Abstract BibTeX arXiv:2209.03534

Code (0)

등록된 구현이 없습니다.

Tasks

Network Pruning

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Learning-to-Transition for Large-scale and High-Order MIMO Detection

2026-08-14 · Yubo Zhang, Yiyao Liu, Xiaodong Wang arxiv

High-order multiple-input multiple-output (MIMO) detection requires efficient search over a large discrete symbol space while producing reliable soft information for channel decoding. This paper develops a learning-to-tr…

Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks

2020-08-19 · Xu Qian, Victor Li, Crews Darren

Second-order information has proven to be very effective in determining the redundancy of neural network weights and activations. Recent paper proposes to use Hessian traces of weights and activations for mixed-precision…

AutoML Deep Reinforcement Learning Quantization

On the Complexity of the Weighted Fused Lasso

2018-01-15 · Jose Bento, Ralph Furmaniak, Surjyendu Ray

The solution path of the 1D fused lasso for an $n$-dimensional input is piecewise linear with $\mathcal{O}(n)$ segments (Hoefling et al. 2010 and Tibshirani et al 2011). However, existing proofs of this bound do not hold…

ANIMC: A Soft Framework for Auto-weighted Noisy and Incomplete Multi-view Clustering

2020-11-20 · Xiang Fang, Yuchong Hu, Pan Zhou, Dapeng Oliver Wu

Multi-view clustering has wide applications in many image processing scenarios. In these scenarios, original image data often contain missing instances and noises, which is ignored by most multi-view clustering methods. …

Clustering Incomplete multi-view clustering regression

Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification

2022-06-24 · Byeonggeun Kim, Seunghan Yang, Jangho Kim, Hyunsin Park 외

While using two-dimensional convolutional neural networks (2D-CNNs) in image processing, it is possible to manipulate domain information using channel statistics, and instance normalization has been a promising way to ge…

Acoustic Scene Classification Domain Generalization Scene Classification