Papers Neural Network Compression
“Neural Network Compression” 태그가 달린 논문 216편 · 필터 해제
Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD
Neural network compression has been an increasingly important subject, not only due to its practical relevance, but also due to its theoretical implications, as there is an explicit connection between compressibility and…
Neural Network CompressionEnd-to-End Neural Network Compression via $\frac{\ell_1}{\ell_2}$ Regularized Latency Surrogates
Neural network (NN) compression via techniques such as pruning, quantization requires setting compression hyperparameters (e.g., number of channels to be pruned, bitwidths for quantization) for each layer either manually…
Neural Architecture SearchNeural Network CompressionQuantizationTransfer LearningUnderstanding the Effect of the Long Tail on Neural Network Compression
Network compression is now a mature sub-field of neural network research: over the last decade, significant progress has been made towards reducing the size of models and speeding up inference, while maintaining the clas…
image-classificationImage ClassificationMemorizationNeural Network CompressionModular Transformers: Compressing Transformers into Modularized Layers for Flexible Efficient Inference
Pre-trained Transformer models like T5 and BART have advanced the state of the art on a wide range of text generation tasks. Compressing these models into smaller ones has become critically important for practical use. C…
DecoderKnowledge DistillationModel CompressionNeural Network Compression+2Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression
This paper introduces a novel theoretical framework for the analysis of vector-valued neural networks through the development of vector-valued variation spaces, a new class of reproducing kernel Banach spaces. These spac…
Multi-Task LearningNeural Network CompressionEvaluation Metrics for DNNs Compression
There is a lot of ongoing research effort into developing different techniques for neural networks compression. However, the community lacks standardised evaluation metrics, which are key to identifying the most suitable…
Neural Network CompressionObjectobject-detectionObject Detection+1How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?
Tensor decompositions have been successfully applied to compress neural networks. The compression algorithms using tensor decompositions commonly minimize the approximation error on the weights. Recent work assumes the a…
Neural Network CompressionTensor DecompositionGuaranteed Quantization Error Computation for Neural Network Model Compression
Neural network model compression techniques can address the computation issue of deep neural networks on embedded devices in industrial systems. The guaranteed output error computation problem for neural network compress…
Model CompressionNeural Network CompressionQuantizationSwiftTron: An Efficient Hardware Accelerator for Quantized Transformers
Transformers' compute-intensive operations pose enormous challenges for their deployment in resource-constrained EdgeAI / tinyML devices. As an established neural network compression technique, quantization reduces the h…
Neural Network CompressionQuantizationWHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks
Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks. Various data-independent criteria, including norm-based and relationship-b…
ClassificationNetwork PruningNeural Network CompressionDepGraph: Towards Any Structural Pruning
Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks. However, the parameter-grouping patterns vary widely across different models, making architecture-specific p…
Network PruningNeural Network CompressionMagnitude and Similarity based Variable Rate Filter Pruning for Efficient Convolution Neural Networks
The superior performance of the recent deep learning models comes at the cost of a significant increase in computational complexity, memory use, and power consumption. Filter pruning is one of the effective neural networ…
Network PruningNeural Network CompressionPD-Quant: Post-Training Quantization based on Prediction Difference Metric
Post-training quantization (PTQ) is a neural network compression technique that converts a full-precision model into a quantized model using lower-precision data types. Although it can help reduce the size and computatio…
Neural Network CompressionQuantizationAcceRL: Policy Acceleration Framework for Deep Reinforcement Learning
Deep reinforcement learning has achieved great success in various fields with its super decision-making ability. However, the policy learning process requires a large amount of training time, causing energy consumption. …
Decision MakingDeep Reinforcement LearningGeneral Reinforcement LearningNeural Network Compression+3Partial Binarization of Neural Networks for Budget-Aware Efficient Learning
Binarization is a powerful compression technique for neural networks, significantly reducing FLOPs, but often results in a significant drop in model performance. To address this issue, partial binarization techniques hav…
BinarizationNeural Network CompressionObject TrackingNeural Network Compression by Joint Sparsity Promotion and Redundancy Reduction
Compression of convolutional neural network models has recently been dominated by pruning approaches. A class of previous works focuses solely on pruning the unimportant filters to achieve network compression. Another im…
Neural Network Compressionspred: Solving $L_1$ Penalty with SGD
We propose to minimize a generic differentiable objective with $L_1$ constraint using a simple reparametrization and straightforward stochastic gradient descent. Our proposal is the direct generalization of previous idea…
Inductive BiasNeural Network CompressionCombining Compressions for Multiplicative Size Scaling on Natural Language Tasks
Quantization, knowledge distillation, and magnitude pruning are among the most popular methods for neural network compression in NLP. Independently, these methods reduce model size and can accelerate inference, but their…
Knowledge DistillationNeural Network CompressionQuantizationQuantized Sparse Weight Decomposition for Neural Network Compression
In this paper, we introduce a novel method of neural network weight compression. In our method, we store weight tensors as sparse, quantized matrix factors, whose product is computed on the fly during inference to genera…
Neural Network CompressionQuantizationSPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks
Recent isotropic networks, such as ConvMixer and vision transformers, have found significant success across visual recognition tasks, matching or outperforming non-isotropic convolutional neural networks (CNNs). Isotropi…
Neural Network Compression