Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
NurtureNet: A Multi-task Video-based Approach for Newborn Anthropometry
Malnutrition among newborns is a top public health concern in developing countries. Identification and subsequent growth monitoring are key to successful interventions. However, this is challenging in rural communities w…
Model CompressionFrom Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Deep neural networks (DNNs) have been widely used in many artificial intelligence (AI) tasks. However, deploying them brings significant challenges due to the huge cost of memory, energy, and computation. To address thes…
Knowledge DistillationModel CompressionQuantizationSurveyLight Field Compression Based on Implicit Neural Representation
Light field, as a new data representation format in multimedia, has the ability to capture both intensity and direction of light rays. However, the additional angular information also brings a large volume of data. Class…
Model CompressionCommunication-Efficient Federated Learning with Adaptive Compression under Dynamic Bandwidth
Federated learning can train models without directly providing local data to the server. However, the frequent updating of the local model brings the problem of large communication overhead. Recently, scholars have achie…
Federated LearningModel CompressionTrio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer
Motivated by the huge success of Transformers in the field of natural language processing (NLP), Vision Transformers (ViTs) have been rapidly developed and achieved remarkable performance in various computer vision tasks…
Efficient ViTsModel CompressionQuantizationIterative Filter Pruning for Concatenation-based CNN Architectures
Model compression and hardware acceleration are essential for the resource-efficient deployment of deep neural networks. Modern object detectors have highly interconnected convolutional layers with concatenations. In thi…
Model CompressionDependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models
The rapid advancement in Large Language Models (LLMs) has markedly enhanced the capabilities of language understanding and generation. However, the substantial model size poses hardware challenges, affecting both memory …
Computational EfficiencyModel CompressionNetwork PruningTorch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design
The development of model compression is continuously motivated by the evolution of various neural network accelerators with ASIC or FPGA. On the algorithm side, the ultimate goal of quantization or pruning is acceleratin…
Model CompressionNeural Network CompressionQuantizationFedGreen: Carbon-aware Federated Learning with Model Size Adaptation
Federated learning (FL) provides a promising collaborative framework to build a model from distributed clients, and this work investigates the carbon emission of the FL process. Cloud and edge servers hosting FL clients …
Federated LearningModel CompressionRapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization
Edge machine learning (ML) enables localized processing of data on devices and is underpinned by deep neural networks (DNNs). However, DNNs cannot be easily run on devices due to their substantial computing, memory and e…
Edge-computingModel CompressionNeural Architecture SearchData-free Knowledge Distillation for Fine-grained Visual Categorization
Data-free knowledge distillation (DFKD) is a promising approach for addressing issues related to model compression, security privacy, and transmission restrictions. Although the existing methods exploiting DFKD have achi…
Data-free Knowledge DistillationFine-Grained Visual CategorizationKnowledge DistillationModel CompressionUnderstanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloads
Among ML operators today, GEneralMatrix Multiplication (GEMM)-based operators are known to be key operators that build the main backbone of ML models. As their computational overhead dominates the overall execution time …
Model CompressionComprehensive Survey of Model Compression and Speed up for Vision Transformers
Vision Transformers (ViT) have marked a paradigm shift in computer vision, outperforming state-of-the-art models across diverse tasks. However, their practical deployment is hampered by high computational and memory dema…
Computational EfficiencyEdge-computingKnowledge DistillationModel Compression+1Structured Model Pruning for Efficient Inference in Computational Pathology
Recent years have seen significant efforts to adopt Artificial Intelligence (AI) in healthcare for various use cases, from computer-aided diagnosis to ICU triage. However, the size of AI models has been rapidly growing d…
Instance SegmentationModel CompressionSemantic SegmentationBayesian Federated Model Compression for Communication and Computation Efficiency
In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variationa…
Bayesian InferenceFederated LearningModel CompressionTransferable and Principled Efficiency for Open-Vocabulary Segmentation
Recent success of pre-trained foundation vision-language models makes Open-Vocabulary Segmentation (OVS) possible. Despite the promising performance, this approach introduces heavy computational overheads for two challen…
Model CompressionSimplifying Two-Stage Detectors for On-Device Inference in Remote Sensing
Deep learning has been successfully applied to object detection from remotely sensed images. Images are typically processed on the ground rather than on-board due to the computation power of the ground system. Such offlo…
Model CompressionObjectobject-detectionObject DetectionMultilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind
Large Language Models (LLMs) have ushered in a new era in Natural Language Processing, but their massive size demands effective compression techniques for practicality. Although numerous model compression techniques have…
Model CompressionOn Linearizing Structured Data in Encoder-Decoder Language Models: Insights from Text-to-SQL
Structured data, prevalent in tables, databases, and knowledge graphs, poses a significant challenge in its representation. With the advent of large language models (LLMs), there has been a shift towards linearization-ba…
DecoderKnowledge GraphsModel CompressionText to SQL+1Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution
Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model. Previous…
Image Super-ResolutionKnowledge DistillationModel CompressionSuper-Resolution