Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training
Deep Multi-agent Reinforcement Learning (MARL) relies on neural networks with numerous parameters in multi-agent scenarios, often incurring substantial computational overhead. Consequently, there is an urgent need to exp…
Model CompressionMulti-agent Reinforcement LearningMaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
Large Language Models (LLMs) are distinguished by their massive parameter counts, which typically result in significant redundancy. This work introduces MaskLLM, a learnable pruning method that establishes Semi-structure…
Large Language ModelModel CompressionNetwork PruningTransfer LearningGeneral Compression Framework for Efficient Transformer Object Tracking
Transformer-based trackers have established a dominant role in the field of visual object tracking. While these trackers exhibit promising performance, their deployment on resource-constrained devices remains challenging…
Model CompressionObjectObject TrackingVisual Object TrackingSearch for Efficient Large Language Models
Large Language Models (LLMs) have long held sway in the realms of artificial intelligence research. Numerous efficient techniques, including weight pruning, quantization, and distillation, have been embraced to compress …
GPUModel CompressionQuantizationEnhancing Knowledge Distillation of Large Language Models through Efficient Multi-Modal Distribution Alignment
Knowledge distillation (KD) is an effective model compression method that can transfer the internal capabilities of large language models (LLMs) to smaller ones. However, the multi-modal probability distribution predicte…
Knowledge DistillationModel CompressionApplications of Knowledge Distillation in Remote Sensing: A Survey
With the ever-growing complexity of models in the field of remote sensing (RS), there is an increasing demand for solutions that balance model accuracy with computational efficiency. Knowledge distillation (KD) has emerg…
Computational EfficiencyInstance SegmentationKnowledge DistillationModel Compression+4ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration
$N{:}M$ sparsity is an emerging model compression method supported by more and more accelerators to speed up sparse matrix multiplication in deep neural networks. Most existing $N{:}M$ sparsity methods compress neural ne…
Model CompressionPrivacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcare
The disparity in healthcare personnel expertise and medical resources across different regions of the world is a pressing social issue. Artificial intelligence technology offers new opportunities to alleviate this issue.…
Data Free QuantizationImage SegmentationModel CompressionPrivacy Preserving+2NVRC: Neural Video Representation Compression
Recent advances in implicit neural representation (INR)-based video coding have demonstrated its potential to compete with both conventional and other learning-based approaches. With INR methods, a neural network is trai…
Model CompressionQuantizationVideo CompressionApplication Specific Compression of Deep Learning Models
Large Deep Learning models are compressed and deployed for specific applications. However, current Deep Learning model compression methods do not utilize the information about the target application. As a result, the com…
Deep LearningModel CompressionNatural Language InferenceParaphrase IdentificationUltron: Enabling Temporal Geometry Compression of 3D Mesh Sequences using Temporal Correspondence and Mesh Deformation
With the advancement of computer vision, dynamic 3D reconstruction techniques have seen significant progress and found applications in various fields. However, these techniques generate large amounts of 3D data sequences…
3D ReconstructionModel CompressionLoCa: Logit Calibration for Knowledge Distillation
Knowledge Distillation (KD), aiming to train a better student model by mimicking the teacher model, plays an important role in model compression. One typical way is to align the output logits. However, we find a common i…
image-classificationImage ClassificationKnowledge DistillationModel Compression+1Efficient Point Cloud Classification via Offline Distillation Framework and Negative-Weight Self-Distillation Technique
The rapid advancement in point cloud processing technologies has significantly increased the demand for efficient and compact models that achieve high-accuracy classification. Knowledge distillation has emerged as a pote…
Data AugmentationKnowledge DistillationModel CompressionPoint Cloud ClassificationFoundations of Large Language Model Compression -- Part 1: Weight Quantization
In recent years, compression of large language models (LLMs) has emerged as an important problem to enable language model deployment on resource-constrained devices, reduce computational costs, and mitigate the environme…
Language ModelingLanguage ModellingLarge Language ModelModel Compression+1Designing Large Foundation Models for Efficient Training and Inference: A Survey
This paper focuses on modern efficient training and inference technologies on foundation models and illustrates them from two perspectives: model and system design. Model and System Design optimize LLM training and infer…
Knowledge DistillationModel CompressionQuantizationSurveyEdge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks
This work evaluates the compression techniques on ConvNeXt models in image classification tasks using the CIFAR-10 dataset. Structured pruning, unstructured pruning, and dynamic quantization methods are evaluated to redu…
Edge-computingimage-classificationImage ClassificationModel Compression+1Hyper-Compression: Model Compression via Hyperfunction
The rapid growth of large models' size has far outpaced that of computing resources. To bridge this gap, encouraged by the parsimonious relationship between genotype and phenotype in the brain's growth and development, w…
modelModel CompressionQuantizationMedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection
Cervical disc herniation (CDH) is a prevalent musculoskeletal disorder that significantly impacts health and requires labor-intensive analysis from experts. Despite advancements in automated detection of medical imaging,…
Knowledge DistillationModel CompressionConvolutional Neural Network Compression Based on Low-Rank Decomposition
Deep neural networks typically impose significant computational loads and memory consumption. Moreover, the large parameters pose constraints on deploying the model on edge devices such as embedded systems. Tensor decomp…
Model CompressionNeural Network CompressionTensor DecompositionVariational autoencoder-based neural network model compression
Variational Autoencoders (VAEs), as a form of deep generative model, have been widely used in recent years, and shown great great peformance in a number of different domains, including image generation and anomaly detect…
Anomaly DetectionImage GenerationmodelModel Compression+1