Papers Neural Network Compression
“Neural Network Compression” 태그가 달린 논문 216편 · 필터 해제
DKM: Differentiable K-Means Clustering Layer for Neural Network Compression
Deep neural network (DNN) model compression for efficient on-device inference is becoming increasingly important to reduce memory requirements and keep user data on-device. To this end, we propose a novel differentiable …
ClusteringModel CompressionNeural Network CompressionBoosting the Convergence of Reinforcement Learning-based Auto-pruning Using Historical Data
Recently, neural network compression schemes like channel pruning have been widely used to reduce the model size and computational complexity of deep neural network (DNN) for applications in power-constrained scenarios s…
Neural Network Compressionreinforcement-learningReinforcement Learning (RL)Transfer LearningData-Driven Low-Rank Neural Network Compression
Despite many modern applications of Deep Neural Networks (DNNs), the large number of parameters in the hidden layers makes them unattractive for deployment on devices with storage capacity constraints. In this paper we p…
Neural Network CompressionHEMP: High-order Entropy Minimization for neural network comPression
We formulate the entropy of a quantized artificial neural network as a differentiable function that can be plugged as a regularization term into the cost function minimized by gradient descent. Our formulation scales eff…
Neural Network CompressionQuantizationVocal Bursts Intensity PredictionJoint Matrix Decomposition for Deep Convolutional Neural Networks Compression
Deep convolutional neural networks (CNNs) with a large number of parameters require intensive computational resources, and thus are hard to be deployed in resource-constrained platforms. Decomposition-based methods, ther…
Efficient Neural NetworkMatrix Factorization / DecompositionNeural Network CompressionExact Backpropagation in Binary Weighted Networks with Group Weight Transformations
Quantization based model compression serves as high performing and fast approach for inference that yields models which are highly compressed when compared to their full-precision floating point counterparts. The most ex…
BinarizationClassification with Binary Weight Networkimage-classificationImage Classification+3Minimally Invasive Surgery for Sparse Neural Networks in Contrastive Manner
With the development of deep learning, neural networks tend to be deeper and larger to achieve good performance. Trained models are more compute-intensive and memory-intensive, which lead to the big challenges on mem…
Knowledge DistillationModel CompressionNeural Network CompressionEfficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
Deep Learning has revolutionized the fields of computer vision, natural language understanding, speech recognition, information retrieval and more. However, with the progressive improvements in deep learning models, thei…
Deep LearningInformation RetrievalModel CompressionNatural Language Understanding+5Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression
Compressing Deep Neural Network (DNN) models to alleviate the storage and computation requirements is essential for practical applications, especially for resource limited devices. Although capable of reducing a reasonab…
Neural Network CompressionHeavy Tails in SGD and Compressibility of Overparametrized Neural Networks
Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks. Recent empirical studies have illustrated that even …
Generalization BoundsNeural Network CompressionTowards Compact CNNs via Collaborative Compression
Channel pruning and tensor decomposition have received extensive attention in convolutional neural network compression. However, these two techniques are traditionally deployed in an isolated manner, leading to significa…
Neural Network CompressionTensor DecompositionData-Free Knowledge Distillation with Soft Targeted Transfer Set Synthesis
Knowledge distillation (KD) has proved to be an effective approach for deep neural network compression, which learns a compact network (student) by transferring the knowledge from a pre-trained, over-parameterized networ…
Data-free Knowledge DistillationKnowledge DistillationNeural Network CompressionSpectral Tensor Train Parameterization of Deep Learning Layers
We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when c…
Deep Learningimage-classificationImage ClassificationImage Generation+1Neural Network Compression for Noisy Storage Devices
Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices. Despite the significant progress in NN model compression, there has been consider…
Model CompressionNeural Network CompressionFAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation
Learning convolutional neural networks (CNNs) with low bitwidth is challenging because performance may drop significantly after quantization. Prior arts often discretize the network weights by carefully tuning hyper-para…
Model CompressionNeural Network CompressionQuantizationScaling Up Exact Neural Network Compression by ReLU Stability
We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable. However, current approaches to determine the stability of …
Neural Network CompressionSparse matrix products for neural network compression
Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investig…
Neural Network CompressionUMEC: Unified model and embedding compression for efficient recommendation systems
The recommendation system (RS) plays an important role in the content recommendation and retrieval scenarios. The core part of the system is the Ranking neural network, which is usually a bottleneck of whole system perfo…
Efficient Neural Networkfeature selectionNeural Network CompressionRecommendation Systems+1Efficient CNN-LSTM based Image Captioning using Neural Network Compression
Modern Neural Networks are eminent in achieving state of the art performance on tasks under Computer Vision, Natural Language Processing and related verticals. However, they are notorious for their voracious memory and c…
DecoderImage CaptioningNeural Network CompressionQuantizationRobustness and Transferability of Universal Attacks on Compressed Models
Neural network compression methods like pruning and quantization are very effective at efficiently deploying Deep Neural Networks (DNNs) on edge devices. However, DNNs remain vulnerable to adversarial examples-inconspicu…
Neural Network CompressionQuantization