Soft Quantization: Model Compression Via Weight Coupling
We show that introducing short-range attractive couplings between the weights of a neural network during training provides a novel avenue for model quantization. These couplings rapidly induce the discretization of a model's weight distribution, and they do so in a mixed-precision manner despite only relying on two additional hyperparameters. We demonstrate that, within an appropriate range of hyperparameters, our "soft quantization'' scheme outperforms histogram-equalized post-training quantization on ResNet-20/CIFAR-10. Soft quantization provides both a new pipeline for the flexible compression of machine learning models and a new tool for investigating the trade-off between compression and generalization in high-dimensional loss landscapes.
Code (0)
등록된 구현이 없습니다.
Tasks
Model CompressionSimilar Papers 제목 키워드 기반
R2 Loss: Range Restriction Loss for Model Compression and Quantization
Model quantization and compression is widely used techniques to reduce usage of computing resource at inference time. While state-of-the-art works have been achieved reasonable accuracy with higher bit such as 4bit or 8b…
ClassificationModel CompressionModel OptimizationQuantizationSoft Weight-Sharing for Neural Network Compression
The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices. This however, conflicts with their computationally, memory and energy intense nature, leading to …
Neural Network CompressionQuantizationCompact Token Representations with Contextual Quantization for Efficient Document Re-ranking
Transformer based re-ranking models can achieve high search relevance through context-aware soft matching of query tokens with document tokens. To alleviate runtime complexity of such inference, previous work has adopted…
QuantizationRe-RankingCompact Token Representations with Contextual Quantization for Efficient Document Re-ranking
Transformer based re-ranking models can achieve high search relevance through context-aware soft matching of query tokens with document tokens. To alleviate runtime complexity of such inference, previous work has adopte…
QuantizationRe-RankingA flexible, extensible software framework for model compression based on the LC algorithm
We propose a software framework based on the ideas of the Learning-Compression (LC) algorithm, that allows a user to compress a neural network or other machine learning model using different compression schemes with mini…
BIG-bench Machine LearningLow-rank compressionModel CompressionNetwork Pruning+1