Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networks
Low-rank compression, a popular model compression technique that produces compact convolutional neural networks (CNNs) with low rankness, has been well-studied in the literature. On the other hand, low-rank training, as …
Low-rank compressionModel CompressionDynamic DNNs and Runtime Management for Efficient Inference on Mobile/Embedded Devices
Deep neural network (DNN) inference is increasingly being executed on mobile and embedded platforms due to several key advantages in latency, privacy and always-on availability. However, due to limited computing resource…
Dynamic neural networksGPUManagementModel CompressionConvolutional Neural Network Compression via Dynamic Parameter Rank Pruning
While Convolutional Neural Networks (CNNs) excel at learning complex latent-space representations, their over-parameterization can lead to overfitting and reduced performance, particularly with limited data. This, alongs…
Model CompressionNeural Network CompressionKnowledge Translation: A New Pathway for Model Compression
Deep learning has witnessed significant advancements in recent years at the cost of increasing training, inference, and model storage overhead. While existing model compression methods strive to reduce the number of mode…
Data AugmentationmodelModel CompressionTranslationFFSplit: Split Feed-Forward Network For Optimizing Accuracy-Efficiency Trade-off in Language Model Inference
The large number of parameters in Pretrained Language Models enhance their performance, but also make them resource-intensive, making it challenging to deploy them on commodity hardware like a single GPU. Due to the memo…
GPULanguage ModelingLanguage ModellingModel CompressionUnderstanding LLMs: A Comprehensive Overview from Training to Inference
The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment with…
Language ModelingLanguage ModellingLarge Language ModelModel Compression+1Retraining-free Model Quantization via One-Shot Weight-Coupling Learning
Quantization is of significance for compressing the over-parameterized deep neural models and deploying them on resource-limited devices. Fixed-precision quantization suffers from performance drop due to the limited nume…
Model CompressionQuantizationSafety and Performance, Why Not Both? Bi-Objective Optimized Model Compression against Heterogeneous Attacks Toward AI Software Deployment
The size of deep learning models in artificial intelligence (AI) software is increasing rapidly, hindering the large-scale deployment on resource-restricted devices (e.g., smartphones). To mitigate this issue, AI softwar…
Inference AttackMembership Inference AttackModel Compressiontest driven developmentUnleashing Channel Potential: Space-Frequency Selection Convolution for SAR Object Detection
Deep Convolutional Neural Networks (DCNNs) have achieved remarkable performance in synthetic aperture radar (SAR) object detection but this comes at the cost of tremendous computational resources partly due to extrac…
feature selectionModel CompressionObjectobject-detection+1Data-Free Quantization via Pseudo-label Filtering
Quantization for model compression can efficiently reduce the network complexity and storage requirement but the original training data is necessary to remedy the performance loss caused by quantization. The Data-Fre…
Data Free QuantizationModel CompressionPseudo LabelPseudo Label Filtering+1Explainability-Driven Leaf Disease Classification Using Adversarial Training and Knowledge Distillation
This work focuses on plant leaf disease classification and explores three crucial aspects: adversarial training, model explainability, and model compression. The models' robustness against adversarial attacks is enhanced…
Adversarial AttackClassificationComputational EfficiencyDecision Making+2DMT: Comprehensive Distillation with Multiple Self-supervised Teachers
Numerous self-supervised learning paradigms, such as contrastive learning and masked image modeling, have been proposed to acquire powerful and general representations from unlabeled data. However, these models are commo…
Contrastive LearningModel CompressionSelf-Supervised LearningIntegrating Fairness and Model Pruning Through Bi-level Optimization
Deep neural networks have achieved exceptional results across a range of applications. As the demand for efficient and sparse deep learning models escalates, the significance of model compression, particularly pruning, i…
FairnessModel CompressionRankDVQA-mini: Knowledge Distillation-Driven Deep Video Quality Assessment
Deep learning-based video quality assessment (deep VQA) has demonstrated significant potential in surpassing conventional metrics, with promising improvements in terms of correlation with human perception. However, the p…
Knowledge DistillationModel CompressionVideo Quality AssessmentVisual Question Answering (VQA)Generative Model-based Feature Knowledge Distillation for Action Recognition
Knowledge distillation (KD), a technique widely employed in computer vision, has emerged as a de facto standard for improving the performance of small neural networks. However, prevailing KD-based approaches in video tas…
Action DetectionAction RecognitionKnowledge DistillationModel Compression+2Unraveling Key Factors of Knowledge Distillation
Knowledge distillation, a technique for model compression and performance enhancement, has gained significant traction in Neural Machine Translation (NMT). However, existing research primarily focuses on empirical applic…
Knowledge DistillationMachine TranslationModel CompressionNMT+1USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models
End-to-end automatic speech recognition (ASR) models have seen revolutionary quality gains with the recent development of large-scale universal speech models (USM). However, deploying these massive USMs is extremely expe…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Model CompressionQuantization+2Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models
Due to the substantial scale of Large Language Models (LLMs), the direct application of conventional compression methodologies proves impractical. The computational demands associated with even minimal gradient updates p…
GPUModel CompressionLarge Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup
The deployment of Large Multimodal Models (LMMs) within AntGroup has significantly advanced multimodal tasks in payment, security, and advertising, notably enhancing advertisement audition tasks in Alipay. However, the d…
Model CompressionNeural Architecture Codesign for Fast Bragg Peak Analysis
We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy. Traditional approaches, notably pseudo-Voigt fitting, demand sign…
AutoMLModel CompressionNetwork PruningNeural Architecture Search+1