Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
An Efficient Real-Time Object Detection Framework on Resource-Constricted Hardware Devices via Software and Hardware Co-design
The fast development of object detection techniques has attracted attention to developing efficient Deep Neural Networks (DNNs). However, the current state-of-the-art DNN models can not provide a balanced solution among …
Model CompressionNeural Network CompressionObjectobject-detection+3Artificial Neural Networks for Photonic Applications: From Algorithms to Implementation
This tutorial-review on applications of artificial neural networks in photonics targets a broad audience, ranging from optical research and engineering communities to computer science and applied mathematics. We focus he…
Model CompressionNeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-level Semantic Slicing
Deep Neural networks (DNNs), extensively applied across diverse disciplines, are characterized by their integrated and monolithic architectures, setting them apart from conventional software systems. This architectural d…
Model CompressionSemantic SimilaritySemantic Textual SimilarityGeneralizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures
Knowledge distillation (KD) is a model compression method that entails training a compact student model to emulate the performance of a more complex teacher model. However, the architectural capacity gap between the two …
Knowledge DistillationModel CompressionTransfer LearningComprehensive Study on Performance Evaluation and Optimization of Model Compression: Bridging Traditional Deep Learning and Large Language Models
Deep learning models have achieved tremendous success in most of the industries in recent years. The evolution of these models has also led to an increase in the model size and energy requirement, making it difficult to …
Deep Learningimage-classificationImage ClassificationKnowledge Distillation+6LORTSAR: Low-Rank Transformer for Skeleton-based Action Recognition
The complexity of state-of-the-art Transformer-based models for skeleton-based action recognition poses significant challenges in terms of computational efficiency and resource utilization. In this paper, we explore the …
Action RecognitionComputational EfficiencyModel CompressionSkeleton Based Action Recognition+1Compressed models are NOT miniature versions of large models
Large neural models are often compressed before deployment. Model compression is necessary for many practical reasons, such as inference latency, memory footprint, and energy consumption. Compressed models are assumed to…
Adversarial AttackModel CompressionMamba-PTQ: Outlier Channels in Recurrent Large Language Models
Modern recurrent layers are emerging as a promising path toward edge deployment of foundation models, especially in the context of large language models (LLMs). Compressing the whole input sequence in a finite-dimensiona…
MambaModel CompressionQuantizationMinimizing PLM-Based Few-Shot Intent Detectors
Recent research has demonstrated the feasibility of training efficient intent detectors based on pre-trained language model~(PLM) with limited labeled data. However, deploying these detectors in resource-constrained envi…
Data AugmentationKnowledge DistillationLanguage ModelingLanguage Modelling+1Inference Optimization of Foundation Models on AI Accelerators
Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries. Industry and research community have witnessed a la…
Inference OptimizationModel CompressionVideo GenerationExplicit-NeRF-QA: A Quality Assessment Database for Explicit NeRF Model Compression
In recent years, Neural Radiance Fields (NeRF) have demonstrated significant advantages in representing and synthesizing 3D scenes. Explicit NeRF models facilitate the practical NeRF applications with faster rendering sp…
Model CompressionNeRFComposable Interventions for Language Models
Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining. But despite a flood of new methods, different types of intervent…
knowledge editingMachine UnlearningModel CompressionQuantizing YOLOv7: A Comprehensive Study
YOLO is a deep neural network (DNN) model presented for robust real-time object detection following the one-stage inference approach. It outperforms other real-time object detectors in terms of speed and accuracy by a wi…
Model Compressionobject-detectionObject DetectionQuantization+1Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
Increasingly, model compression techniques enable large language models (LLMs) to be deployed in real-world applications. As a result of this momentum towards local deployment, compressed LLMs will interact with a large …
Language ModelingLanguage ModellingModel CompressionQuantizationAMD: Automatic Multi-step Distillation of Large-scale Vision Models
Transformer-based architectures have become the de-facto standard models for diverse vision tasks owing to their superior performance. As the size of the models continues to scale up, model distillation becomes extremely…
image-classificationImage ClassificationKnowledge DistillationModel CompressionThe Impact of Quantization and Pruning on Deep Reinforcement Learning Models
Deep reinforcement learning (DRL) has achieved remarkable success across various domains, such as video games, robotics, and, recently, large language models. However, the computational costs and memory requirements of D…
Deep Reinforcement LearningModel CompressionNeural Network CompressionQuantizationEfficient DNN-Powered Software with Fair Sparse Models
With the emergence of the Software 3.0 era, there is a growing trend of compressing and integrating large models into software systems, with significant societal implications. Regrettably, in numerous instances, model co…
FairnessModel CompressionMLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
Knowledge distillation is an effective technique for pre-trained language model compression. Although existing knowledge distillation methods perform well for the most typical model BERT, they could be further improved i…
Extractive Question-AnsweringKnowledge DistillationLanguage ModelingLanguage Modelling+2FoldGPT: Simple and Effective Large Language Model Compression Scheme
The demand for deploying large language models(LLMs) on mobile devices continues to increase, driven by escalating data security concerns and cloud costs. However, network bandwidth and memory limitations pose challenges…
Language ModelingLanguage ModellingLarge Language ModelModel CompressionMCNC: Manifold Constrained Network Compression
The outstanding performance of large foundational models across diverse tasks-from computer vision to speech and natural language processing-has significantly increased their demand. However, storing and transmitting the…
Model CompressionQuantization