Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
InstaGeo: Compute-Efficient Geospatial Machine Learning from Data to Deployment
Open-access multispectral imagery from missions like Landsat 8-9 and Sentinel-2 has fueled the development of geospatial foundation models (GFMs) for humanitarian and environmental applications. Yet, their deployment rem…
Model CompressionFederated Split Learning for Resource-Constrained Robots in Industrial IoT: Framework Comparison, Optimization Strategies, and Future Directions
Federated split learning (FedSL) has emerged as a promising paradigm for enabling collaborative intelligence in industrial Internet of Things (IoT) systems, particularly in smart factories where data privacy, communicati…
Model CompressionDownsized and Compromised?: Assessing the Faithfulness of Model Compression
In real-world applications, computational constraints often require transforming large models into smaller, more efficient versions through model compression. While these techniques aim to reduce size and computational c…
Model CompressionConditional Pseudo-Supervised Contrast for Data-Free Knowledge Distillation
Data-free knowledge distillation~(DFKD) is an effective manner to solve model compression and transmission restrictions while retaining privacy protection, which has attracted extensive attention in recent years. Current…
Data-free Knowledge DistillationContrastive LearningModel CompressionCHORD: Customizing Hybrid-precision On-device Model for Sequential Recommendation with Device-cloud Collaboration
With the advancement of mobile device capabilities, deploying reranking models directly on devices has become feasible, enabling real-time contextual recommendations. When migrating models from cloud to devices, resource…
Sequential RecommendationModel CompressionSAGE: Streaming Agreement-Driven Gradient Sketches for Representative Subset Selection
Training modern neural networks on large datasets is computationally and energy intensive. We present SAGE, a streaming data-subset selection method that maintains a compact Frequent Directions (FD) sketch of gradient ge…
Model CompressionInterpret, prune and distill Donut : towards lightweight VLMs for VQA on document
Recent advances in Visually-rich Document Understanding rely on large Vision-Language Models like Donut, which perform document-level Visual Question Answering without Optical Character Recognition. Despite their effecti…
Visual Question AnsweringKnowledge DistillationModel CompressionBALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression
Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on a…
Model CompressionCLQ: Cross-Layer Guided Orthogonal-based Quantization for Diffusion Transformers
Visual generation quality has been greatly promoted with the rapid advances in diffusion transformers (DiTs), which is attributed to the scaling of model size and complexity. However, these attributions also hinder the p…
Model CompressionVideo GenerationImage GenerationThe Hidden Costs of Translation Accuracy: Distillation, Quantization, and Environmental Impact
The rapid expansion of large language models (LLMs) has heightened concerns about their computational and environmental costs. This study investigates the trade-offs between translation quality and efficiency by comparin…
Machine TranslationModel CompressionIIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
High-order numerical methods enhance Transformer performance in tasks like NLP and CV, but introduce a performance-efficiency trade-off due to increased computational overhead. Our analysis reveals that conventional effi…
Model CompressionRetoVLA: Reusing Register Tokens for Spatial Reasoning in Vision-Language-Action Models
Vision-Language-Action (VLA) models have demonstrated robust performance across diverse robotic tasks. However, their high memory and computational demands often limit real-time deployment. While existing model compressi…
Spatial ReasoningModel CompressionTowards Adapting Federated & Quantum Machine Learning for Network Intrusion Detection: A Survey
This survey explores the integration of Federated Learning (FL) with Network Intrusion Detection Systems (NIDS), with particular emphasis on deep learning and quantum machine learning approaches. FL enables collaborative…
Network Intrusion DetectionQuantum Machine LearningFederated LearningModel CompressionSpeculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
The immense model sizes of large language models (LLMs) challenge deployment on memory-limited consumer GPUs. Although model compression and parameter offloading are common strategies to address memory limitations, compr…
Model CompressionMaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
The evolution of quantization and mixed-precision techniques has unlocked new possibilities for enhancing the speed and energy efficiency of NNs. Several recent studies indicate that adapting precision levels across diff…
Model CompressionGAPrune: Gradient-Alignment Pruning for Domain-Aware Embeddings
Domain-specific embedding models have shown promise for applications that require specialized semantic understanding, such as coding agents and financial retrieval systems, often achieving higher performance gains than g…
Model CompressionStrategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
Distributed and federated learning are essential paradigms for training models across decentralized data sources while preserving privacy, yet communication overhead remains a major bottleneck. This dissertation explores…
Federated LearningModel CompressionCompressing CNN models for resource-constrained systems by channel and layer pruning
Convolutional Neural Networks (CNNs) have achieved significant breakthroughs in various fields. However, these advancements have led to a substantial increase in the complexity and size of these networks. This poses a ch…
Model CompressionIntegrating Pruning with Quantization for Efficient Deep Neural Networks Compression
Deep Neural Networks (DNNs) have achieved significant advances in a wide range of applications. However, their deployment on resource-constrained devices remains a challenge due to the large number of layers and paramete…
Model CompressionDaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
Large language models (LLMs) excel in general tasks but struggle with domain-specific ones, requiring fine-tuning with specific data. With many open-source LLMs available, selecting the best model for fine-tuning downstr…
Reading ComprehensionModel Compression