Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization
Mixture-of-Experts (MoE) models have emerged as a cornerstone of large-scale deep learning by efficiently distributing computation and enhancing performance. However, their unique architecture-characterized by sparse exp…
Mixture-of-ExpertsModel CompressionQuantizationAttribution-guided Pruning for Compression, Circuit Discovery, and Targeted Correction in LLMs
Large Language Models (LLMs) are central to many contemporary AI applications, yet their extensive parameter counts pose significant challenges for deployment in memory- and compute-constrained environments. Recent works…
Model CompressionSimple is what you need for efficient and accurate medical image segmentation
While modern segmentation models often prioritize performance over practicality, we advocate a design philosophy prioritizing simplicity and efficiency, and attempted high performance segmentation model design. This pape…
feature selectionImage SegmentationMedical Image SegmentationModel Compression+3FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation
Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years, as it avoids computationally intensive model retraining. Nevertheless, current PTQ methods for …
Model CompressionQuantizationPost-Training Quantization for Video Matting
Video matting is crucial for applications such as film production and virtual reality, yet deploying its computationally intensive models on resource-constrained devices presents challenges. Quantization is a key techniq…
Image MattingModel CompressionOptical Flow EstimationQuantization+1Advances in Small-Footprint Keyword Spotting: A Comprehensive Review of Efficient Models and Algorithms
Small-Footprint Keyword Spotting (SF-KWS) has gained popularity in today's landscape of smart voice-activated devices, smartphones, and Internet of Things (IoT) applications. This surge is attributed to the advancements …
Automatic Speech RecognitionKeyword SpottingModel CompressionSmall-Footprint Keyword Spotting+2AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descent
To address the enormous size of Large Language Models (LLMs), model compression methods, such as quantization and pruning, are often deployed, especially on edge devices. In this work, we focus on layer-wise post-trainin…
Model CompressionQuantizationStructured Pruning and Quantization for Learned Image Compression
The high computational costs associated with large deep learning models significantly hinder their practical deployment. Model pruning has been widely explored in deep learning literature to reduce their computational bu…
image-classificationImage ClassificationImage CompressionModel Compression+4INSIGHT: A Survey of In-Network Systems for Intelligent, High-Efficiency AI and Topology Optimization
In-network computation represents a transformative approach to addressing the escalating demands of Artificial Intelligence (AI) workloads on network infrastructure. By leveraging the processing capabilities of network d…
Federated LearningIntrusion DetectionModel CompressionSmooth Model Compression without Fine-Tuning
Compressing and pruning large machine learning models has become a critical step towards their deployment in real-world applications. Standard pruning and compression techniques are typically designed without taking the …
modelModel CompressionFLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression
Large Language Models (LLMs) have enabled remarkable progress in natural language processing, yet their high computational and memory demands pose challenges for deployment in resource-constrained environments. Although …
Language ModelingLanguage ModellingLarge Language ModelModel CompressionEffective and Efficient One-pass Compression of Speech Foundation Models Using Sparsity-aware Self-pinching Gates
This paper presents a novel approach for speech foundation models compression that tightly integrates model pruning and parameter update into a single stage. Highly compact layer-level tied self-pinching gates each conta…
Model CompressionSmall Language Models: Architectures, Techniques, Evaluation, Problems and Future Adaptation
Small Language Models (SLMs) have gained substantial attention due to their ability to execute diverse language tasks successfully while using fewer computer resources. These models are particularly ideal for deployment …
Model CompressionQuantizationResSVD: Residual Compensated SVD for Large Language Model Compression
Large language models (LLMs) have demonstrated impressive capabilities in a wide range of downstream natural language processing tasks. Nevertheless, their considerable sizes and memory demands hinder practical deploymen…
Language ModelingLanguage ModellingLarge Language ModelModel CompressionPangu Light: Weight Re-Initialization for Pruning and Accelerating LLMs
Large Language Models (LLMs) deliver state-of-the-art capabilities across numerous tasks, but their immense size and inference costs pose significant computational challenges for practical deployment. While structured pr…
Model CompressionEfficient Speech Translation through Model Compression and Knowledge Distillation
Efficient deployment of large audio-language models for speech translation remains challenging due to their significant computational requirements. In this paper, we address this challenge through our system submissions …
Knowledge DistillationModel CompressionQuantizationTranslationTensorization is a powerful but underexplored tool for compression and interpretability of neural networks
Tensorizing a neural network involves reshaping some or all of its dense weight matrices into higher-order tensors and approximating them using low-rank tensor network decompositions. This technique has shown promise as …
Deep LearningModel CompressionKnowledge Grafting of Large Language Models
Cross-capability transfer is a key challenge in large language model (LLM) research, with applications in multi-task integration, model compression, and continual learning. Recent works like FuseLLM and FuseChat have dem…
Continual LearningKnowledge DistillationLarge Language ModelModel Compression+1Efficient and Workload-Aware LLM Serving via Runtime Layer Swapping and KV Cache Resizing
Efficiently serving large language models (LLMs) under dynamic and bursty workloads remains a key challenge for real-world deployment. Existing serving frameworks and static model compression techniques fail to adapt to …
Model CompressionQuantizationSchedulingMaking deep neural networks work for medical audio: representation, compression and domain adaptation
This thesis addresses the technical challenges of applying machine learning to understand and interpret medical audio signals. The sounds of our lungs, heart, and voice convey vital information about our health. Yet, in …
Domain AdaptationModel CompressionTensor DecompositionTransfer Learning