RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal of inference runtime acceleration. In this paper, we introduce a novel data-free pruning protocol RED++. Only requiring a trained neural network, and not specific to DNN architecture, we exploit an adaptive data-free scalar hashing which exhibits redundancies among neuron weight values. We study the theoretical and empirical guarantees on the preservation of the accuracy from the hashing as well as the expected pruning ratio resulting from the exploitation of said redundancies. We propose a novel data-free pruning technique of DNN layers which removes the input-wise redundant operations. This algorithm is straightforward, parallelizable and offers novel perspective on DNN pruning by shifting the burden of large computation to efficient memory access and allocation. We provide theoretical guarantees on RED++ performance and empirically demonstrate its superiority over other data-free pruning methods and its competitiveness with data-driven ones on ResNets, MobileNets and EfficientNets.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Design of Rate-Splitting Multiple Access With Imperfect CSI for Cell-Free MIMO Systems
Rate-Splitting Multiple Access (RSMA) for multi-user downlink operates by splitting the message for each user equipment (UE) into a private message and a set of common messages, which are simultaneously transmitted by me…
Robust DesignSplitting Convolutional Neural Network Structures for Efficient Inference
For convolutional neural networks (CNNs) that have a large volume of input data, memory management becomes a major concern. Memory cost reduction can be an effective way to deal with these problems that can be realized t…
ManagementData-free Backdoor Removal based on Channel Lipschitzness
Recent studies have shown that Deep Neural Networks (DNNs) are vulnerable to the backdoor attacks, which leads to malicious behaviors of DNNs when specific triggers are attached to the input images. It was further demons…
Rate-Splitting assisted Massive Machine-Type Communications in Cell-Free Massive MIMO
This letter focuses on integrating rate-splitting multiple-access (RSMA) with time-division-duplex Cell-free Massive MIMO (multiple-input multiple-output) for massive machine-type communications. Due to the large number …
Channel Pruning In Quantization-aware Training: An Adaptive Projection-gradient Descent-shrinkage-splitting Method
We propose an adaptive projection-gradient descent-shrinkage-splitting method (APGDSSM) to integrate penalty based channel pruning into quantization-aware training (QAT). APGDSSM concurrently searches weights in both the…
Quantization