Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
EPIM: Efficient Processing-In-Memory Accelerators based on Epitome
The utilization of large-scale neural networks on Processing-In-Memory (PIM) accelerators encounters challenges due to constrained on-chip memory capacity. To tackle this issue, current works explore model compression al…
Model CompressionNeural Architecture SearchQuantizationSupervised domain adaptation for building extraction from off-nadir aerial images
Building extraction $-$ needed for inventory management and planning of urban environment $-$ is affected by the misalignment between labels and off-nadir source imagery in training data. Teacher-Student learning of nois…
Domain AdaptationEarth ObservationExperimental DesignKnowledge Distillation+3What is Lost in Knowledge Distillation?
Deep neural networks (DNNs) have improved NLP tasks significantly, but training and maintaining such networks could be costly. Model compression techniques, such as, knowledge distillation (KD), have been proposed to add…
Knowledge DistillationModel CompressionAsymmetric Masked Distillation for Pre-Training Small Foundation Models
Self-supervised foundation models have shown great potential in computer vision thanks to the pre-training paradigm of masked autoencoding. Scale is a primary factor influencing the performance of these foundation models…
Action ClassificationAction RecognitionImage ClassificationKnowledge Distillation+1Data-Free Distillation of Language Model by Text-to-Text Transfer
Data-Free Knowledge Distillation (DFKD) plays a vital role in compressing the model when original training data is unavailable. Previous works for DFKD in NLP mainly focus on distilling encoder-only structures like BERT …
Data-free Knowledge DistillationDiversityKnowledge DistillationLanguage Modeling+5Divergent Token Metrics: Measuring degradation to prune away LLM components -- and optimize quantization
Large Language Models (LLMs) have reshaped natural language processing with their impressive capabilities. However, their ever-increasing size has raised concerns about their effective deployment and the need for LLM com…
ManagementModel CompressionQuantizationText GenerationRetrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression
Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, the massive size of these models poses huge challenges for their deplo…
Language ModelingLanguage ModellingLarge Language ModelModel Compression+2LXMERT Model Compression for Visual Question Answering
Large-scale pretrained models such as LXMERT are becoming popular for learning cross-modal representations on text-image pairs for vision-language tasks. According to the lottery ticket hypothesis, NLP and computer visio…
modelModel CompressionVisual Question AnsweringVisual Question Answering (VQA)Data-Free Knowledge Distillation Using Adversarially Perturbed OpenGL Shader Images
Knowledge distillation (KD) has been a popular and effective method for model compression. One important assumption of KD is that the original training dataset is always available. However, this is not always the case du…
Data AugmentationData-free Knowledge DistillationKnowledge DistillationModel CompressionUSDC: Unified Static and Dynamic Compression for Visual Transformer
Visual Transformers have achieved great success in almost all vision tasks, such as classification, detection, and so on. However, the model complexity and the inference speed of the visual transformers hinder their depl…
Model CompressionIn defense of parameter sharing for model-compression
When considering a model architecture, there are several ways to reduce its memory footprint. Historically, popular approaches included selecting smaller architectures and creating sparse networks through pruning. More r…
Model CompressionEfficient Apple Maturity and Damage Assessment: A Lightweight Detection Model with GAN and Attention Mechanism
This study proposes a method based on lightweight convolutional neural networks (CNN) and generative adversarial networks (GAN) for apple ripeness and damage level detection tasks. Initially, a lightweight CNN model is d…
Decision MakingModel Compressionobject-detectionObject DetectionWhat do larger image classifiers memorise?
The success of modern neural networks has prompted study of the connection between memorisation and generalisation: overparameterised models generalise well, despite being able to perfectly fit (memorise) completely rand…
image-classificationImage ClassificationKnowledge DistillationMemorization+1A Corrected Expected Improvement Acquisition Function Under Noisy Observations
Sequential maximization of expected improvement (EI) is one of the most widely used policies in Bayesian optimization because of its simplicity and ability to handle noisy observations. In particular, the improvement fun…
Bayesian OptimizationModel CompressionAccelerating Machine Learning Primitives on Commodity Hardware
Sliding Window Sum algorithms have been successfully used for training and inference of Deep Neural Networks. We have shown before how both pooling and convolution 1-D primitives could be expressed as sliding sums and ev…
CPUModel CompressionModel Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences
On-device machine learning (ML) promises to improve the privacy, responsiveness, and proliferation of new, intelligent user experiences by moving ML computation onto everyday personal devices. However, today's large ML m…
Model CompressionSparse Deep Learning for Time Series Data: Theory and Applications
Sparse deep learning has become a popular technique for improving the performance of deep neural networks in areas such as uncertainty quantification, variable selection, and large-scale network compression. However, mos…
Deep LearningModel CompressionTime SeriesUncertainty Quantification+1Robustness-Guided Image Synthesis for Data-Free Quantization
Quantization has emerged as a promising direction for model compression. Recently, data-free quantization has been widely studied as a promising method to avoid privacy concerns, which synthesizes images as an alternativ…
Data Free QuantizationDiversityImage GenerationModel Compression+1ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Models
Large Vision-Language Models (LVLMs) can understand the world comprehensively by integrating rich information from different modalities, achieving remarkable advancements on various multimodal downstream tasks. However, …
Model CompressionSweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained with a single task in mind. Due to high co…
Model CompressionText Summarization