Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Is Finer Better? The Limits of Microscaling Formats in Large Language Models
Microscaling data formats leverage per-block tensor quantization to enable aggressive model compression with limited loss in accuracy. Unlocking their potential for efficient training and inference necessitates hardware-…
Model CompressionPocketDVDNet: Realtime Video Denoising for Real Camera Noise
Live video denoising under realistic, multi-component sensor noise remains challenging for applications such as autofocus, autonomous driving, and surveillance. We propose PocketDVDNet, a lightweight video denoiser devel…
Knowledge DistillationAutonomous DrivingModel CompressionVideo DenoisingDistilling Time Series Foundation Models for Efficient Forecasting
Time Series foundation models (TSFMs) deliver strong forecasting performance through large-scale pretraining, but their large parameter sizes make deployment costly. While knowledge distillation offers a natural and effe…
Time Series ForecastingKnowledge DistillationModel CompressionLPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data
This paper addresses the limitations of current vision-based rail defect detection methods, including high computational complexity, excessive parameter counts, and suboptimal accuracy. We propose a Lightweight Pyramid C…
Model CompressionSoftware-Hardware Co-optimization for Modular E2E AV Paradigm: A Unified Framework of Optimization Approaches, Simulation Environment and Evaluation Metrics
Modular end-to-end (ME2E) autonomous driving paradigms combine modular interpretability with global optimization capability and have demonstrated strong performance. However, existing studies mainly focus on accuracy imp…
Autonomous DrivingModel CompressionGarbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruning
Large Language Models (LLMs) are known to contain significant redundancy, yet a systematic explanation for why certain components, particularly in higher layers, are more redundant has remained elusive. In this work, we …
Model CompressionASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition
Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures. In the context of Automatic Speech Recognition (ASR), previous studies have …
Knowledge DistillationSpeech RecognitionModel CompressionSparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models
Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments. Low-…
Model CompressionLearning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer
Knowledge distillation has proven effective for model compression by transferring knowledge from a larger network called the teacher to a smaller network called the student. Current knowledge distillation in time series …
Knowledge DistillationTime Series AnalysisModel CompressionSparse Knowledge Distillation: A Mathematical Framework for Probability-Domain Temperature Scaling and Multi-Stage Compression
We develop a unified theoretical framework for sparse knowledge distillation based on probability-domain softening operators. While the equivalence $p^{1/T} \propto \mathrm{softmax}(z/T)$ is well known, our contribution …
Knowledge DistillationModel CompressionLightweight Transformer Architectures for Edge Devices in Real-Time Applications
The deployment of transformer-based models on resource-constrained edge devices represents a critical challenge in enabling real-time artificial intelligence applications. This comprehensive survey examines lightweight t…
Neural Architecture SearchKnowledge DistillationModel CompressionOptimizing LSTM Neural Networks for Resource-Constrained Retail Sales Forecasting: A Model Compression Study
Standard LSTM(Long Short-Term Memory) neural networks provide accurate predictions for sales data in the retail industry, but require a lot of computing power. It can be challenging especially for mid to small retail ind…
Model CompressionKnowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
Reasoning over temporal knowledge graphs (TKGs) is fundamental to improving the efficiency and reliability of intelligent decision-making systems and has become a key technological foundation for future artificial intell…
Computational EfficiencyKnowledge DistillationModel CompressionKnowledge GraphsClean-GS: Semantic Mask-Guided Pruning for 3D Gaussian Splatting
3D Gaussian Splatting produces high-quality scene reconstructions but generates hundreds of thousands of spurious Gaussians (floaters) scattered throughout the environment. These artifacts obscure objects of interest and…
Model CompressionCan Large Language Models Still Explain Themselves? Investigating the Impact of Quantization on Self-Explanations
Quantization is widely used to accelerate inference and streamline the deployment of large language models (LLMs), yet its effects on self-explanations (SEs) remain unexplored. SEs, generated by LLMs to justify their own…
Model CompressionEvaluating the Impact of Compression Techniques on the Robustness of CNNs under Natural Corruptions
Compressed deep learning models are crucial for deploying computer vision systems on resource-constrained devices. However, model compression may affect robustness, especially under natural corruption. Therefore, it is i…
Model CompressionFAST-IDS: A Fast Two-Stage Intrusion Detection System with Hybrid Compression for Real-Time Threat Detection in Connected and Autonomous Vehicles
We have implemented a multi-stage IDS for CAVs that can be deployed to resourec-constrained environments after hybrid model compression.
Autonomous VehiclesIntrusion DetectionModel CompressionSAP: Syntactic Attention Pruning for Transformer-based Language Models
This paper introduces Syntactic Attention Pruning (SAP), a novel method for effectively pruning attention heads in Transformer models. Unlike conventional approaches that rely solely on mathematical analysis of model wei…
Model CompressionScaling Laws for Energy Efficiency of Local LLMs
Deploying local large language models and vision-language models on edge devices requires balancing accuracy with constrained computational and energy budgets. Although graphics processors dominate modern artificial-inte…
Model CompressionTOGGLE: Temporal Logic-Guided Large Language Model Compression for Edge
Large Language Models (LLMs) deliver exceptional performance across natural language tasks but demand substantial computational resources, limiting their deployment on resource-constrained edge devices. Existing compress…
Model Compression