Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
CoDeQ: End-to-End Joint Model Compression with Dead-Zone Quantizer for High-Sparsity and Low-Precision Networks
While joint pruning--quantization is theoretically superior to sequential application, current joint methods rely on auxiliary procedures outside the training loop for finding compression parameters. This reliance adds e…
Model CompressionDisentangled and Distilled Encoder for Out-of-Distribution Reasoning with Rademacher Guarantees
Recently, the disentangled latent space of a variational autoencoder (VAE) has been used to reason about multi-label out-of-distribution (OOD) test samples that are derived from different distributions than training samp…
Model CompressionHPM-KD: Hierarchical Progressive Multi-Teacher Framework for Knowledge Distillation and Efficient Model Compression
Knowledge Distillation (KD) has emerged as a promising technique for model compression but faces critical limitations: (1) sensitivity to hyperparameters requiring extensive manual tuning, (2) capacity gap when distillin…
Knowledge DistillationModel CompressionSkewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devices
In recent years, high-performance computer vision models have achieved remarkable success in medical imaging, with some skin lesion classification systems even surpassing dermatology specialists in diagnostic accuracy. H…
Skin Lesion ClassificationFederated LearningModel CompressionHOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability
Deep learning models have achieved remarkable success across various domains, yet their learned representations and decision-making processes remain largely opaque and hard to interpret. This work introduces HOLE (Homolo…
Model CompressionQuantization Blindspots: How Model Compression Breaks Backdoor Defenses
Backdoor attacks embed input-dependent malicious behavior into neural networks while preserving high clean accuracy, making them a persistent threat for deployed ML systems. At the same time, real-world deployments almos…
Model CompressionNeural expressiveness for beyond importance model compression
Neural Network Pruning has been established as driving force in the exploration of memory and energy efficient solutions with high throughput both during training and at test time. In this paper, we introduce a novel cri…
Model CompressionObject DetectionNetwork PruningSHRP: Specialized Head Routing and Pruning for Efficient Encoder Compression
Transformer encoders are widely deployed in large-scale web services for natural language understanding tasks such as text classification, semantic retrieval, and content ranking. However, their high inference latency an…
Natural Language UnderstandingText ClassificationSemantic RetrievalModel CompressionFirst On-Orbit Demonstration of a Geospatial Foundation Model
Geospatial foundation models (GeoFMs) promise broad generalisation capacity for Earth observation (EO) tasks, particularly under data-limited conditions. However, their large size poses a barrier to deployment on resourc…
Model CompressionDomain AdaptationKV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference
Long-context Large Language Models (LLMs) face significant memory bottlenecks during inference due to the linear growth of key-value (KV) cache with sequence length. While individual optimization techniques like KV cache…
Model CompressionEnhancing Trustworthiness with Mixed Precision: Benchmarks, Opportunities, and Challenges
Large language models (LLMs) have shown promising performance across various tasks. However, their autoregressive decoding process poses significant challenges for efficient deployment on existing AI hardware. Quantizati…
Adversarial RobustnessModel CompressionIDAP++: Advancing Divergence-Based Pruning via Filter-Level and Layer-Level Optimization
This paper presents a novel approach to neural network compression that addresses redundancy at both the filter and architectural levels through a unified framework grounded in information flow analysis. Building on the …
Neural Network CompressionModel CompressionUncertainty-Aware Dual-Student Knowledge Distillation for Efficient Image Classification
Knowledge distillation has emerged as a powerful technique for model compression, enabling the transfer of knowledge from large teacher networks to compact student models. However, traditional knowledge distillation meth…
Knowledge DistillationImage ClassificationModel CompressionPost-Pruning Accuracy Recovery via Data-Free Knowledge Distillation
Model pruning is a widely adopted technique to reduce the computational complexity and memory footprint of Deep Neural Networks (DNNs). However, global unstructured pruning often leads to significant degradation in accur…
Data-free Knowledge DistillationModel CompressionA Systematic Study of Compression Ordering for Large Language Models
Large Language Models (LLMs) require substantial computational resources, making model compression essential for efficient deployment in constrained environments. Among the dominant compression techniques: knowledge dist…
Knowledge DistillationModel CompressionExtreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression
The demand for edge AI in vision-language tasks requires models that achieve real-time performance on resource-constrained devices with limited power and memory. This paper proposes two adaptive compression techniques --…
Event-based visionModel CompressionChange DetectionActDistill: General Action-Guided Self-Derived Distillation for Efficient Vision-Language-Action Models
Recent Vision-Language-Action (VLA) models have shown impressive flexibility and generalization, yet their deployment in robotic manipulation remains limited by heavy computational overhead and inference latency. In this…
Model CompressionSparOA: Sparse and Operator-aware Hybrid Scheduling for Edge DNN Inference
The resource demands of deep neural network (DNN) models introduce significant performance challenges, especially when deployed on resource-constrained edge devices. Existing solutions like model compression often sacrif…
Reinforcement LearningModel CompressionE$^3$-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models
With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruning methods struggle to simultaneously ad…
Knowledge DistillationModel CompressionFT-NCFM: An Influence-Aware Data Distillation Framework for Efficient VLA Models
The powerful generalization of Vision-Language-Action (VLA) models is bottlenecked by their heavy reliance on massive, redundant, and unevenly valued datasets, hindering their widespread application. Existing model-centr…
Model Compression