Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Budgeted LoRA: Distillation as Structured Compute Allocation for Efficient Inference
We study distillation for large language models under explicit compute constraints, with the goal of producing student models that are not only cheaper to train, but structurally efficient at inference time. While prior …
Model CompressionComplexity Horizons of Compressed Models in Analog Circuit Analysis
The deployment of Large Language Models (LLMs) for specialized engineering domains, such as circuit analysis, often faces a trade-off between reasoning accuracy and computational efficiency. Traditional evaluation method…
Computational EfficiencyModel CompressionSwiftChannel: Algorithm-Hardware Co-Design for Deep Learning-Based 5G Channel Estimation
Channel estimation is crucial in 5G communication networks for optimizing transmission parameters and ensuring reliable, high-speed communication. However, the use of multiple-input and multiple-output (MIMO) and millime…
Knowledge DistillationModel CompressionModel Compression with Exact Budget Constraints via Riemannian Manifolds
Assigning one of K options to each of N groups under a total cost budget is a recurring problem in efficient AI, including mixed-precision quantization, non-uniform pruning, and expert selection. The objective, typically…
Model CompressionRevisiting the Effectiveness of LLM Pruning for Test-Time Scaling
Large Language Models (LLMs) now exhibit remarkable reasoning capabilities through test-time compute scaling (TTS), with impressive performance across math and coding benchmarks. In parallel, research in model compressio…
Model CompressionFed-DLoRA: Efficient Wireless Federated Learning with Dynamic Low-Rank Adaptation
Federated learning (FL) offers a promising distributed learning paradigm for internet of vehicles (IoV) applications. However, it faces challenges from communication overhead and dynamic environments. Model compression t…
Federated LearningModel CompressionS-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models
General audio foundation models have recently achieved remarkable progress, enabling strong performance across diverse tasks. However, state-of-the-art models remain extremely large, often with hundreds of millions of pa…
Knowledge DistillationModel CompressionEfficient VQ-QAT and Mixed Vector/Linear quantized Neural Networks
In this work, we developed and tested 3 techniques for vector quantization (VQ) based model weight compression. To mitigate codebook collapse and enable end-to-end training, we adopted cosine similarity-based assignment.…
Neural Architecture SearchModel CompressionHierarchical Spatio-Channel Clustering for Efficient Model Compression in Medical Image Analysis
Convolutional neural networks (CNNs) have become increasingly difficult to deploy in resource-constrained environments due to their large memory and computational requirements. Although low-rank compression methods can r…
Model CompressionSelective Depthwise Separable Convolution for Lightweight Joint Source-Channel Coding in Wireless Image Transmission
Depthwise separable convolutional (DSConv) layers have been successfully applied to deep learning (DL)-based joint source-channel coding (JSCC) schemes to reduce computational complexity. However, a systematic investigat…
Model CompressionEfficient Video Diffusion Models: Advancements and Challenges
Video diffusion models have rapidly become the dominant paradigm for high-fidelity generative video synthesis, but their practical deployment remains constrained by severe inference costs. Compared with image generation,…
Model CompressionImage GenerationExploiting Correlations in Federated Learning: Opportunities and Practical Limitations
The communication bottleneck in federated learning (FL) has spurred extensive research into techniques to reduce the volume of data exchanged between client devices and the central parameter server. In this paper, we sys…
Federated LearningModel CompressionOFA-Diffusion Compression: Compressing Diffusion Model in One-Shot Manner
The Diffusion Probabilistic Model (DPM) achieves remarkable performance in image generation, while its increasing parameter size and computational overhead hinder its deployment in practical applications. To improve this…
Model CompressionImage GenerationLong-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation
Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window attention (SWA). However, SWA inevitably loses distant history during l…
Computational EfficiencyModel CompressionVideo GenerationDeFakeQ: Enabling Real-Time Deepfake Detection on Edge Devices via Adaptive Bidirectional Quantization
Deepfake detection has become a fundamental component of modern media forensics. Despite significant progress in detection accuracy, most existing methods remain computationally intensive and parameter-heavy, limiting th…
DeepFake DetectionModel CompressionDP-OPD: Differentially Private On-Policy Distillation for Language Models
Large language models (LLMs) are increasingly adapted to proprietary and domain-specific corpora that contain sensitive information, creating a tension between formal privacy guarantees and efficient deployment through m…
Knowledge DistillationModel CompressionText GenerationSLaB: Sparse-Lowrank-Binary Decomposition for Efficient Large Language Models
The rapid growth of large language models (LLMs) presents significant deployment challenges due to their massive computational and memory demands. While model compression, such as network pruning, offers potential soluti…
Model CompressionNetwork PruningLow-Rank Compression of Pretrained Models via Randomized Subspace Iteration
The massive scale of pretrained models has made efficient compression essential for practical deployment. Low-rank decomposition based on the singular value decomposition (SVD) provides a principled approach for model re…
Model CompressionMulti-Aspect Knowledge Distillation for Language Model with Low-rank Factorization
Knowledge distillation is an effective technique for pre-trained language model compression. However, existing methods only focus on the knowledge distribution among layers, which may cause the loss of fine-grained infor…
Knowledge DistillationModel CompressionLiteInception: A Lightweight and Interpretable Deep Learning Framework for General Aviation Fault Diagnosis
General aviation fault diagnosis and efficient maintenance are critical to flight safety; however, deploying deep learning models on resource-constrained edge devices poses dual challenges in computational capacity and i…
Knowledge DistillationModel CompressionFault Diagnosis