Structured Deep Neural Network Pruning via Matrix Pivoting
Deep Neural Networks (DNNs) are the key to the state-of-the-art machine vision, sensor fusion and audio/video signal processing. Unfortunately, their computation complexity and tight resource constraints on the Edge make them hard to leverage on mobile, embedded and IoT devices. Due to great diversity of Edge devices, DNN designers have to take into account the hardware platform and application requirements during network training. In this work we introduce pruning via matrix pivoting as a way to improve network pruning by compromising between the design flexibility of architecture-oblivious and performance efficiency of architecture-aware pruning, the two dominant techniques for obtaining resource-efficient DNNs. We also describe local and global network optimization techniques for efficient implementation of the resulting pruned networks. In combination, the proposed pruning and implementation result in close to linear speed up with the reduction of network coefficients during pruning.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityNetwork PruningSensor FusionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models
The rapid growth of Large Language Models has driven demand for effective model compression techniques to reduce memory and computation costs. Low-rank pruning has gained attention for its tensor coherence and GPU compat…
GPUModel CompressionTight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation
The unstructured sparsity after pruning poses a challenge to the efficient implementation of deep learning models in existing regular architectures like systolic arrays. On the other hand, coarse-grained structured pruni…
Model CompressionTowards Perturbation-Induced Static Pivoting on GPU-Based Linear Solvers
Linear system solving is a key tool for computational power system studies, e.g., optimal power flow, transmission switching, or unit commitment. CPU-based linear system solver speeds, however, have saturated in recent y…
CPUGPULearnable Permutation for Structured Sparsity on Transformer Models
Structured sparsity has emerged as a popular model pruning technique, widely adopted in various architectures, including CNNs, Transformer models, and especially large language models (LLMs) in recent years. A promising …
Structured Pruning of Large Language Models
Large language models have recently achieved state of the art performance across a wide variety of natural language tasks. Meanwhile, the size of these models and their latency have significantly increased, which makes t…
Language ModelingLanguage ModellingModel CompressionWord Embeddings