Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Principled Approximation Methods for Efficient and Scalable Deep Learning
Recent progress in deep learning has been driven by increasingly larger models. However, their computational and energy demands have grown proportionally, creating significant barriers to their deployment and to a wider …
Neural Architecture SearchImage ClassificationModel CompressionThe Role of Teacher Calibration in Knowledge Distillation
Knowledge Distillation (KD) has emerged as an effective model compression technique in deep learning, enabling the transfer of knowledge from a large teacher model to a compact student model. While KD has demonstrated si…
Knowledge DistillationModel CompressionLess Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions
Model compression through post-training pruning offers a way to reduce model size and computational requirements without significantly impacting model performance. However, the effect of pruning on the fairness of LLM-ge…
Model CompressionInterpreting the Effects of Quantization on LLMs
Quantization offers a practical solution to deploy LLMs in resource-constraint environments. However, its impact on internal representations remains understudied, raising questions about the reliability of quantized mode…
Model CompressionAn Empirical Study of Knowledge Distillation for Code Understanding Tasks
Pre-trained language models (PLMs) have emerged as powerful tools for code understanding. However, deploying these PLMs in large-scale applications faces practical challenges due to their computational intensity and infe…
Knowledge DistillationModel CompressionCALR: Corrective Adaptive Low-Rank Decomposition for Efficient Large Language Model Layer Compression
Large Language Models (LLMs) present significant deployment challenges due to their immense size and computational requirements. Model compression techniques are essential for making these models practical for resource-c…
Model CompressionSynthetic Adaptive Guided Embeddings (SAGE): A Novel Knowledge Distillation Method
Model distillation enables the transfer of knowledge from large-scale models to compact student models, facilitating deployment in resource-constrained environments. However, conventional distillation approaches often su…
Dimensionality ReductionKnowledge DistillationModel CompressionData AugmentationOne Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression
Pruning is a core technique for compressing neural networks to improve computational efficiency. This process is typically approached in two ways: one-shot pruning, which involves a single pass of training and pruning, a…
Computational EfficiencyModel CompressionNovel Parasitic Dual-Scale Modeling for Efficient and Accurate Multilingual Speech Translation
Recent advancements in speech-to-text translation have led to the development of multilingual models capable of handling multiple language pairs simultaneously. However, these unified models often suffer from large param…
Speech-to-Text TranslationKnowledge DistillationModel CompressionLLM Compression: How Far Can We Go in Balancing Size and Performance?
Quantization is an essential and popular technique for improving the accessibility of large language models (LLMs) by reducing memory usage and computational costs while maintaining performance. In this study, we apply 4…
Mathematical ReasoningInformation RetrievalQuestion AnsweringModel CompressionUnpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models
Sharpness-Aware Minimization (SAM) has been proven to be an effective optimization technique for improving generalization in overparameterized models. While prior works have explored the implicit regularization of SAM in…
parameter-efficient fine-tuningModel CompressionLabel-Noise Resistant Learning via Optimal Brain Damage Masking
Noisy labels are inevitable in real-world multimedia applications. Due to the strong memorization capacity of deep neural networks, these noisy labels cause significant performance degradation. Existing noise-robust meth…
Model CompressionLLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
Large Vision-Language Models (VLMs) exhibit impressive multi-modal capabilities but suffer from prohibitive computational and memory demands, due to their long visual token sequences and massive parameter sizes. To addre…
Model CompressionTowards Scalable Lottery Ticket Networks using Genetic Algorithms
Building modern deep learning systems that are not just effective but also efficient requires rethinking established paradigms for model training and neural architecture design. Instead of adapting highly overparameteriz…
Multi-class ClassificationModel CompressionCognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning
Efficient control of prosthetic limbs via non-invasive brain-computer interfaces (BCIs) requires advanced EEG processing, including pre-filtering, feature extraction, and action prediction, performed in real time on edge…
Computational EfficiencyModel CompressionCOMponent-Aware Pruning for Accelerated Control Tasks in Latent Space Models
The rapid growth of resource-constrained mobile platforms, including mobile robots, wearable systems, and Internet-of-Things devices, has increased the demand for computationally efficient neural network controllers (NNC…
Reinforcement LearningModel CompressionPerceive-Sample-Compress: Towards Real-Time 3D Gaussian Splatting
Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated remarkable capabilities in real-time and photorealistic novel view synthesis. However, traditional 3DGS representations often struggle with large-scale sc…
Novel View SynthesisModel CompressionMoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
The Mixture-of-Experts (MoE) architecture has become a predominant paradigm for scaling large language models (LLMs). Despite offering strong performance and computational efficiency, large MoE-based LLMs like DeepSeek-V…
Computational EfficiencyModel CompressionTask complexity shapes internal representations and robustness in neural networks
Neural networks excel across a wide range of tasks, yet remain black boxes. In particular, how their internal representations are shaped by the complexity of the input data and the problems they solve remains obscure. In…
Model CompressionS$^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation
Diffusion transformers have emerged as the mainstream paradigm for video generation models. However, the use of up to billions of parameters incurs significant computational costs. Quantization offers a promising solutio…
Model CompressionVideo Generation