Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies
Deploying foundation models in embodied edge systems is fundamentally a systems problem, not just a problem of model compression. Real-time control must operate within strict size, weight, and power constraints, where me…
Model CompressionSafety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retraining
Safety prompts constitute an interpretable layer of defense against jailbreak attacks in vision-language models (VLMs); however, their efficacy is constrained by the models' latent structural responsiveness. We observe t…
Model CompressionSoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression
Large language models (LLMs) have demonstrated impressive capabilities across various tasks, but the billion-scale parameters pose deployment challenges. Although existing methods attempt to reduce the scale of LLMs, the…
Model CompressionLeech Lattice Vector Quantization for Efficient LLM Compression
Scalar quantization of large language models (LLMs) is fundamentally limited by information-theoretic bounds. While vector quantization (VQ) overcomes these limits by encoding blocks of parameters jointly, practical impl…
Model Compression3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
Sparse plus Low-Rank $(\mathbf{S} + \mathbf{LR})$ decomposition of Large Language Models (LLMs) has emerged as a promising direction in model compression, aiming to decompose pre-trained model weights into a sum of spars…
Model CompressionGRAIL: Post-hoc Compensation by Linear Reconstruction for Compressed Networks
Structured deep model compression methods are hardware-friendly and substantially reduce memory and inference costs. However, under aggressive compression, the resulting accuracy degradation often necessitates post-compr…
Model CompressionBound to Disagree: Generalization Bounds via Certifiable Surrogates
Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes. In this paper, we tackle these issues by providing new disagreement-based certificates for the…
Model CompressionHybridINR-PCGC: Hybrid Lossless Point Cloud Geometry Compression Bridging Pretrained Model and Implicit Neural Representation
Learning-based point cloud compression presents superior performance to handcrafted codecs. However, pretrained-based methods, which are based on end-to-end training and expected to generalize to all the potential sample…
Representation LearningModel CompressionTT-SEAL: TTD-Aware Selective Encryption for Adversarially-Robust and Low-Latency Edge AI
Cloud-edge AI must jointly satisfy model compression and security under tight device budgets. While Tensor-Train Decomposition (TTD) shrinks on-device models, prior selective-encryption studies largely assume dense weigh…
Model CompressionDropping Anchor and Spherical Harmonics for Sparse-view Gaussian Splatting
Recent 3D Gaussian Splatting (3DGS) Dropout methods address overfitting under sparse-view conditions by randomly nullifying Gaussian opacities. However, we identify a neighbor compensation effect in these approaches: dro…
Model CompressionCut Less, Fold More: Model Compression through the Lens of Projection Geometry
Compressing neural networks without retraining is vital for deployment at scale. We study calibration-free compression through the lens of projection geometry: structured pruning is an axis-aligned projection, whereas mo…
Model CompressionSPQ: An Ensemble Technique for Large Language Model Compression
This study presents an ensemble technique, SPQ (SVD-Pruning-Quantization), for large language model (LLM) compression that combines variance-retained singular value decomposition (SVD), activation-based pruning, and post…
Model CompressionSign Lock-In: Randomly Initialized Weight Signs Persist and Bottleneck Sub-Bit Model Compression
Sub-bit model compression targets storage below one bit per weight; as magnitudes are aggressively compressed, the sign bit becomes a fixed-cost bottleneck. Across Transformers, CNNs, and MLPs, learned sign matrices resi…
Model CompressionAlgorithmic Simplification of Neural Networks with Mosaic-of-Motifs
Large-scale deep learning models are well-suited for compression. Across a variety of tasks, methods like pruning, quantization, and knowledge distillation have been used to achieve massive reductions in model parameters…
Knowledge DistillationModel CompressionExplainability-Inspired Layer-Wise Pruning of Deep Neural Networks for Efficient Object Detection
Deep neural networks (DNNs) have achieved remarkable success in object detection tasks, but their increasing complexity poses significant challenges for deployment on resource-constrained platforms. While model compressi…
Model CompressionObject DetectionA Comprehensive Survey on Deep Learning-Based LiDAR Super-Resolution for Autonomous Driving
LiDAR sensors are often considered essential for autonomous driving, but high-resolution sensors remain expensive while affordable low-resolution sensors produce sparse point clouds that miss critical details. LiDAR supe…
Autonomous DrivingModel CompressionPoint CloudsInterpretable Vision Transformers in Image Classification via SVDA
Vision Transformers (ViTs) have achieved state-of-the-art performance in image classification, yet their attention mechanisms often remain opaque and exhibit dense, non-structured behaviors. In this work, we adapt our pr…
Image ClassificationModel CompressionROCKET: Rapid Optimization via Calibration-guided Knapsack Enhanced Truncation for Efficient Model Compression
We present ROCKET, a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines. Operating under a glob…
Model CompressionBeyond Student: An Asymmetric Network for Neural Network Inheritance
Knowledge Distillation (KD) has emerged as a powerful technique for model compression, enabling lightweight student networks to benefit from the performance of redundant teacher networks. However, the inherent capacity g…
Knowledge DistillationModel CompressionLittleBit-2: Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment
We identify the Spectral Energy Gain in extreme model compression, where low-rank binary approximations outperform tiny-rank floating-point baselines for heavy-tailed spectra. However, prior attempts fail to realize this…
Model Compression