Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
FlattenGPT: Depth Compression for Transformer with Layer Flattening
Recent works have indicated redundancy across transformer blocks, prompting the research of depth compression to prune less crucial blocks. However, current ways of entire-block pruning suffer from risks of discarding me…
Model CompressionUniComp: A Unified Evaluation of Large Language Model Compression via Pruning, Quantization, and Distillation
Model compression is increasingly essential for deploying large language models (LLMs), yet existing comparative studies largely focus on pruning and quantization evaluated primarily on knowledge-centric benchmarks. Thus…
Knowledge DistillationModel CompressionAngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression
This technical report introduces AngelSlim, a comprehensive and versatile toolkit for large model compression developed by the Tencent Hunyuan team. By consolidating cutting-edge algorithms, including quantization, specu…
Model CompressionNeural Networks With Dense Weights Are Not Universal Approximators
We investigate the approximation capabilities of dense neural networks. While universal approximation theorems establish that sufficiently large architectures can approximate arbitrary continuous functions if there are n…
Model CompressionNanoFLUX: Distillation-Driven Compression of Large Text-to-Image Generation Models for Mobile Devices
While large-scale text-to-image diffusion models continue to improve in visual quality, their increasing scale has widened the gap between state-of-the-art models and on-device solutions. To address this gap, we introduc…
Text-to-Image GenerationModel CompressionGreedy-Gnorm: A Gradient Matrix Norm-Based Alternative to Attention Entropy for Head Pruning
Attention head pruning has emerged as an effective technique for transformer model compression, an increasingly important goal in the era of Green AI. However, existing pruning methods often rely on static importance sco…
Model CompressionLSGQuant: Layer-Sensitivity Guided Quantization for One-Step Diffusion Real-World Video Super-Resolution
One-Step Diffusion Models have demonstrated promising capability and fast inference in video super-resolution (VSR) for real-world. Nevertheless, the substantial model size and high computational cost of Diffusion Transf…
Video Super-ResolutionModel CompressionFARTrack: Fast Autoregressive Visual Tracking with High Performance
Inference speed and tracking performance are two critical evaluation metrics in the field of visual tracking. However, high-performance trackers often suffer from slow processing speeds, making them impractical for deplo…
Model CompressionVisual TrackingQuantization-Aware Regularizers for Deep Neural Networks Compression
Deep Neural Networks reached state-of-the-art performance across numerous domains, but this progress has come at the cost of increasingly large and over-parameterized models, posing serious challenges for deployment on r…
Model CompressionQVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
The advent of Vision-Language-Action (VLA) models represents a significant leap for embodied intelligence, yet their immense computational demands critically hinder deployment on resource-constrained robotic platforms. I…
Model CompressionEntropy Reveals Block Importance in Masked Self-Supervised Vision Transformers
Masked self-supervised vision transformers have become a dominant pretraining paradigm, yet their substantial model size poses significant challenges for resource-constrained deployment and efficient transfer learning. A…
Transfer LearningModel CompressionResponsible AI in Business
Artificial intelligence (AI) and Machine Learning (ML) have moved from research and pilot projects into everyday business operations, with generative AI accelerating adoption across processes, products, and services. Thi…
Model CompressionDomain AdaptationEfficient Deep Learning for Medical Imaging: Bridging the Gap Between High-Performance AI and Clinical Deployment
Deep learning has revolutionized medical image analysis, playing a vital role in modern clinical applications. However, the deployment of large-scale models in real-world clinical settings remains challenging due to high…
Knowledge DistillationModel CompressionFloat8@2bits: Entropy Coding Enables Data-Free Model Compression
Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibility but suffer from functional collapse…
Model CompressionAgile Reinforcement Learning through Separable Neural Architecture and Applications
Deep reinforcement learning (RL) is increasingly deployed in resource-constrained environments, yet go-to function approximators - multilayer perceptrons (MLPs) - are often parameter-inefficient due to an imperfect induc…
Reinforcement LearningModel CompressionSoft Quantization: Model Compression Via Weight Coupling
We show that introducing short-range attractive couplings between the weights of a neural network during training provides a novel avenue for model quantization. These couplings rapidly induce the discretization of a mod…
Model CompressionHeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error. The root cause lies…
Model CompressionMeeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKIT
Enterprise LLM deployment faces a critical scalability challenge: organizations must optimize models systematically to scale AI initiatives within constrained compute budgets, yet the specialized expertise required for m…
Model CompressionLoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation
Post-training quantization (PTQ) enables effective model compression while preserving relatively high accuracy. Current weight-only PTQ methods primarily focus on the challenging sub-3-bit regime, where approaches often …
Model CompressionComponent-Aware Pruning Framework for Neural Network Controllers via Gradient-Based Importance Estimation
The transition from monolithic to multi-component neural architectures in advanced neural network controllers poses substantial challenges due to the high computational complexity of the latter. Conventional model compre…
Model Compression