Papers Neural Network Compression
“Neural Network Compression” 태그가 달린 논문 216편 · 필터 해제
Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compression
This paper proposes a novel matrix quantization method, Binary Quadratic Quantization (BQQ). In contrast to conventional first-order quantization approaches, such as uniform quantization and binary coding quantization, t…
Neural Network CompressionOne-Bit Quantization for Random Features Models
Recent advances in neural networks have led to significant computational and memory demands, spurring interest in one-bit weight compression to enable efficient inference on resource-constrained devices. However, the the…
Neural Network CompressionIPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning
Importance-based structured pruning overwhelmingly relies on filter magnitude. This proxy is fundamentally flawed: due to scale invariance, functionally identical filters can receive arbitrarily different importance scor…
Neural Network CompressionLinearity-based neural network compression
In neural network compression, most current methods reduce unnecessary parameters by measuring importance and redundancy. To augment already highly optimized existing solutions, we propose linearity-based compression as …
Efficient Neural NetworkNeural Network CompressionMUC-G4: Minimal Unsat Core-Guided Incremental Verification for Deep Neural Network Compression
The rapid development of deep learning has led to challenges in deploying neural networks on edge devices, mainly due to their high memory and runtime complexity. Network compression techniques, such as quantization and …
Neural Network CompressionQuantizationIs Quantum Optimization Ready? An Effort Towards Neural Network Compression using Adiabatic Quantum Computing
Quantum optimization is the most mature quantum computing technology to date, providing a promising approach towards efficiently solving complex combinatorial problems. Methods such as adiabatic quantum computing (AQC) h…
Model CompressionNeural Network CompressionQuantizationCertified Neural Approximations of Nonlinear Dynamics
Neural networks hold great potential to act as approximate models of nonlinear dynamical systems, with the resulting neural approximations enabling verification and control of such systems. However, in safety-critical co…
Neural Network CompressionLow-Rank Matrix Approximation for Neural Network Compression
Deep Neural Networks (DNNs) have encountered an emerging deployment challenge due to large and expensive memory and computation requirements. In this paper, we present a new Adaptive-Rank Singular Value Decomposition (AR…
Model CompressionNeural Network CompressionGranQ: Granular Zero-Shot Quantization with Channel-Wise Activation Scaling in QAT
Zero-shot quantization (ZSQ) enables neural network compression without original training data, making it a promising solution for restricted data access scenarios. To compensate for the lack of data, recent ZSQ methods …
Neural Network CompressionQuantizationStabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrix
Quantization-Aware Training (QAT) is one of the prevailing neural network compression solutions. However, its stability has been challenged for yielding deteriorating performances as the quantization error is inevitable.…
Neural Network CompressionQuantizationCompression of Site-Specific Deep Neural Networks for Massive MIMO Precoding
The deployment of deep learning (DL) models for precoding in massive multiple-input multiple-output (mMIMO) systems is often constrained by high computational demands and energy consumption. In this paper, we investigate…
Neural Architecture SearchNeural Network CompressionQuantizationA Novel Structure-Agnostic Multi-Objective Approach for Weight-Sharing Compression in Deep Neural Networks
Deep neural networks suffer from storing millions and billions of weights in memory post-training, making challenging memory-intensive models to deploy on embedded devices. The weight-sharing technique is one of the popu…
Neural Network CompressionQuantizationWhat is Left After Distillation? How Knowledge Transfer Impacts Fairness and Bias
Knowledge Distillation is a commonly used Deep Neural Network (DNN) compression method, which often maintains overall generalization performance. However, we show that even for balanced image classification datasets, suc…
Age/UnbiasedFairnessGender Bias Detectionimage-classification+6Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching
Knowledge Distillation (KD) has emerged as a pivotal technique for neural network compression and performance enhancement. Most KD methods aim to transfer dark knowledge from a cumbersome teacher model to a lightweight s…
Knowledge DistillationNeural Network CompressionLanguage Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models
Despite the widespread use of statistical prior models in various fields, such models for neural network gradients have long been overlooked. The inherent challenge stems from their high-dimensional structures and comple…
Neural Network CompressionQuantizationAdaptive Error-Bounded Hierarchical Matrices for Efficient Neural Network Compression
This paper introduces a dynamic, error-bounded hierarchical matrix (H-matrix) compression method tailored for Physics-Informed Neural Networks (PINNs). The proposed approach reduces the computational complexity and memor…
Efficient Neural NetworkNeural Network CompressionQuantizationUnified Framework for Neural Network Compression via Decomposition and Optimal Rank Selection
Despite their high accuracy, complex neural networks demand significant computational resources, posing challenges for deployment on resource-constrained devices such as mobile phones and embedded systems. Compression al…
Neural Network CompressionTensor DecompositionTropNNC: Structured Neural Network Compression Using Tropical Geometry
We present TropNNC, a framework for compressing neural networks with linear and convolutional layers and ReLU activations. TropNNC is a structured compression framework based on a geometrical approach to machine/deep lea…
Neural Network CompressionConvolutional Neural Network Compression Based on Low-Rank Decomposition
Deep neural networks typically impose significant computational loads and memory consumption. Moreover, the large parameters pose constraints on deploying the model on edge devices such as embedded systems. Tensor decomp…
Model CompressionNeural Network CompressionTensor DecompositionCondensed Sample-Guided Model Inversion for Knowledge Distillation
Knowledge distillation (KD) is a key element in neural network compression that allows knowledge transfer from a pre-trained teacher model to a more compact student model. KD relies on access to the training dataset, whi…
Knowledge DistillationmodelNeural Network CompressionTransfer Learning