Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Lightweight Design and Optimization methods for DCNNs: Progress and Futures
Lightweight design, as a key approach to mitigate disparity between computational requirements of deep learning models and hardware performance, plays a pivotal role in advancing application of deep learning technologies…
Autonomous DrivingDeep LearningModel CompressionPhilosophySemantics Prompting Data-Free Quantization for Low-Bit Vision Transformers
Data-free quantization (DFQ), which facilitates model quantization without real data to address increasing concerns about data security, has garnered significant attention within the model compression community. Recently…
Data Free QuantizationModel CompressionQuantizationMix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN
Large Language Models (LLMs) have achieved remarkable success, yet recent findings reveal that their deeper layers often contribute minimally and can be pruned without affecting overall performance. While some view this …
Model CompressionDeploying Foundation Model Powered Agent Services: A Survey
Foundation model (FM) powered agent services are regarded as a promising solution to develop intelligent and personalized applications for advancing toward Artificial General Intelligence (AGI). To achieve high reliabili…
modelModel CompressionSurveyToken ReductionRemoteTrimmer: Adaptive Structural Pruning for Remote Sensing Image Classification
Since high resolution remote sensing image classification often requires a relatively high computation complexity, lightweight models tend to be practical and efficient. Model pruning is an effective method for model com…
Classificationimage-classificationImage ClassificationModel Compression+2TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs
Specializing large language models (LLMs) for local deployment in domain-specific use cases is necessary for strong performance while meeting latency and privacy constraints. However, conventional task-specific adaptatio…
Model CompressionQuantizationCan Students Beyond The Teacher? Distilling Knowledge from Teacher's Bias
Knowledge distillation (KD) is a model compression technique that transfers knowledge from a large teacher model to a smaller student model to enhance its performance. Existing methods often assume that the student model…
Knowledge DistillationModel CompressionActivation Sparsity Opportunities for Compressing General Large Language Models
Deploying local AI models, such as Large Language Models (LLMs), to edge devices can substantially enhance devices' independent capabilities, alleviate the server's burden, and lower the response time. Owing to these tre…
Model CompressionOptimising TinyML with Quantization and Distillation of Transformer and Mamba Models for Indoor Localisation on Edge Devices
This paper proposes small and efficient machine learning models (TinyML) for resource-constrained edge devices, specifically for on-device indoor localisation. Typical approaches for indoor localisation rely on centralis…
Knowledge DistillationMambaModel CompressionQuantizationLow-Rank Correction for Quantized LLMs
We consider the problem of model compression for Large Language Models (LLMs) at post-training time, where the task is to compress a well-trained model using only a small set of calibration input data. In this work, we i…
Model CompressionQuantizationVQ4ALL: Efficient Neural Network Representation via a Universal Codebook
The rapid growth of the big neural network models puts forward new requirements for lightweight network representation methods. The traditional methods based on model compression have achieved great success, especially V…
Density EstimationEfficient Neural NetworkModel CompressionCompression for Better: A General and Stable Lossless Compression Framework
This work focus on how to stabilize and lossless model compression, aiming to reduce model complexity and enhance efficiency without sacrificing performance due to compression errors. A key challenge is effectively lever…
Computational EfficiencyModel CompressionQuantizationLossless Model Compression via Joint Low-Rank Factorization Optimization
Low-rank factorization is a popular model compression technique that minimizes the error $\delta$ between approximated and original weight matrices. Despite achieving performances close to the original models when $\delt…
Model CompressionModel OptimizationTrimming Down Large Spiking Vision Transformers via Heterogeneous Quantization Search
Spiking Neural Networks (SNNs) are amenable to deployment on edge devices and neuromorphic hardware due to their lower dissipation. Recently, SNN-based transformers have garnered significant interest, incorporating atten…
Model CompressionQuantizationEfficient Model Compression Techniques with FishLeg
In many domains, the most successful AI models tend to be the largest, indeed often too large to be handled by AI players with limited computational resources. To mitigate this, a number of compression methods have been …
Meta-LearningmodelModel CompressionCPTQuant -- A Novel Mixed Precision Post-Training Quantization Techniques for Large Language Models
Large language models have transformed the comprehension and generation of natural language tasks, but they come with substantial memory and computational requirements. Quantization techniques have emerged as a promising…
Language ModelingLanguage ModellingModel CompressionQuantization+1Individual Content and Motion Dynamics Preserved Pruning for Video Diffusion Models
The high computational cost and slow inference time are major obstacles to deploying the video diffusion model (VDM) in practical applications. To overcome this, we introduce a new Video Diffusion Model Compression appro…
Model CompressionVideo GenerationEfficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion
As text-to-image models grow increasingly powerful and complex, their burgeoning size presents a significant obstacle to widespread adoption, especially on resource-constrained devices. This paper presents a pioneering s…
Image GenerationModel CompressionFaithful Label-free Knowledge Distillation
Knowledge distillation approaches are model compression techniques, with the goal of training a highly performant student model by using a teacher network that is larger or contains a different inductive bias. These appr…
Inductive BiasKnowledge DistillationModel CompressionOut-of-Distribution DetectionTaQ-DiT: Time-aware Quantization for Diffusion Transformers
Transformer-based diffusion models, dubbed Diffusion Transformers (DiTs), have achieved state-of-the-art performance in image and video generation tasks. However, their large model size and slow inference speed limit the…
DenoisingModel CompressionQuantizationVideo Generation