Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures
Pruning - that is, setting a significant subset of the parameters of a neural network to zero - is one of the most popular methods of model compression. Yet, several recent works have raised the issue that pruning may in…
Model CompressionNetwork PruningDeep Collective Knowledge Distillation
Many existing studies on knowledge distillation have focused on methods in which a student model mimics a teacher model well. Simply imitating the teacher's knowledge, however, is not sufficient for the student to surpas…
Knowledge DistillationModel CompressionLearning Accurate Performance Predictors for Ultrafast Automated Model Compression
In this paper, we propose an ultrafast automated model compression framework called SeerNet for flexible network deployment. Conventional non-differen-tiable methods discretely search the desirable compression policy bas…
image-classificationImage ClassificationModel Compressionobject-detection+2Structured Pruning for Multi-Task Deep Neural Networks
Although multi-task deep neural network (DNN) models have computation and storage benefits over individual single-task DNN models, they can be further optimized via model compression. Numerous structured pruning methods …
Model CompressionSurrogate Lagrangian Relaxation: A Path To Retrain-free Deep Neural Network Pruning
Network pruning is a widely used technique to reduce computation cost and model size for deep neural networks. However, the typical three-stage pipeline significantly increases the overall training time. In this paper, w…
image-classificationImage ClassificationLane DetectionModel Compression+3oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain between 3.8 and 24.3 times faster models witho…
Knowledge DistillationModel CompressionQuantizationQuestion Answering+1Information-Theoretic GAN Compression with Variational Energy-based Model
We propose an information-theoretic knowledge distillation approach for the compression of generative adversarial networks, which aims to maximize the mutual information between teacher and student networks via a variati…
Image EnhancementKnowledge DistillationModel CompressionA Multi-objective Complex Network Pruning Framework Based on Divide-and-conquer and Global Performance Impairment Ranking
Model compression plays a vital role in the practical deployment of deep neural networks (DNNs), and evolutionary multi-objective (EMO) pruning is an essential tool in balancing the compression rate and performance of th…
Model CompressionNetwork PruningTetra-AML: Automatic Machine Learning via Tensor Networks
Neural networks have revolutionized many aspects of society but in the era of huge models with billions of parameters, optimizing and deploying them for commercial applications can require significant computational and f…
Bayesian OptimizationHyperparameter OptimizationModel CompressionNeural Architecture Search+3Towards Accurate Post-Training Quantization for Vision Transformer
Vision transformer emerges as a potential architecture for vision tasks. However, the intense computation and non-negligible delay hinder its application in the real world. As a widespread model compression technique, ex…
Model CompressionQuantizationExploring Turkish Speech Recognition via Hybrid CTC/Attention Architecture and Multi-feature Fusion Network
In recent years, End-to-End speech recognition technology based on deep learning has developed rapidly. Due to the lack of Turkish speech data, the performance of Turkish speech recognition system is poor. Firstly, this …
Model Compressionspeech-recognitionSpeech RecognitionLow Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training
Deep neural networks have achieved great success in many data processing applications. However, the high computational complexity and storage cost makes deep learning hard to be used on resource-constrained devices, and …
Model CompressionQuantizationPerformance-aware Approximation of Global Channel Pruning for Multitask CNNs
Global channel pruning (GCP) aims to remove a subset of channels (filters) across different layers from a deep model without hurting the performance. Previous works focus on either single task model pruning or simply ada…
Model CompressionThe Tiny Time-series Transformer: Low-latency High-throughput Classification of Astronomical Transients using Deep Model Compression
A new golden age in astronomy is upon us, dominated by data. Large astronomical surveys are broadcasting unprecedented rates of information, demanding machine learning as a critical component in modern scientific pipelin…
AstronomyModel CompressionTime SeriesA Contrastive Knowledge Transfer Framework for Model Compression and Transfer Learning
Knowledge Transfer (KT) achieves competitive performance and is widely used for image classification tasks in model compression and transfer learning. Existing KT works transfer the information from a large model ("teach…
image-classificationImage ClassificationModel CompressionTransfer LearningI3D: Transformer architectures with input-dependent dynamic depth for speech recognition
Transformer-based end-to-end speech recognition has achieved great success. However, the large footprint and computational overhead make it difficult to deploy these models in some real-world applications. Model compress…
Model Compressionspeech-recognitionSpeech RecognitionR2 Loss: Range Restriction Loss for Model Compression and Quantization
Model quantization and compression is widely used techniques to reduce usage of computing resource at inference time. While state-of-the-art works have been achieved reasonable accuracy with higher bit such as 4bit or 8b…
ClassificationModel CompressionModel OptimizationQuantizationOTOV2: Automatic, Generic, User-Friendly
The existing model compression methods via structured pruning typically require complicated multi-stage procedures. Each individual stage necessitates numerous engineering efforts and domain-knowledge from the end-users …
Model CompressionOn Model Compression for Neural Networks: Framework, Algorithm, and Convergence Guarantee
Model compression is a crucial part of deploying neural networks (NNs), especially when the memory and storage of computing devices are limited in many applications. This paper focuses on two model compression techniques…
Image ClassificationModel CompressionTensor DecompositionGreener yet Powerful: Taming Large Code Generation Models with Quantization
ML-powered code generation aims to assist developers to write code in a more productive manner, by intelligently generating code blocks based on natural language prompts. Recently, large pretrained deep learning models h…
Code GenerationCode SummarizationModel CompressionQuantization