Papers Model Compression
“Model Compression” 태그가 달린 논문 1,632편 · 필터 해제
Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation
We investigate whether knowledge distillation (KD) from multiple heterogeneous teacher models can enhance the generation of transferable adversarial examples. A lightweight student model is trained using two KD strategie…
Knowledge DistillationModel CompressionTowards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks
Although LLMs have attained significant success in high-resource languages, their capacity in low-resource linguistic environments like Kannada and Arabic is not yet fully understood. This work benchmarking the performan…
Cross-Lingual TransferModel CompressionModel Compression Engine for Wearable Devices Skin Cancer Diagnosis
Skin cancer is one of the most prevalent and preventable types of cancer, yet its early detection remains a challenge, particularly in resource-limited settings where access to specialized healthcare is scarce. This stud…
Binary ClassificationModel CompressionTransfer LearningCompLeak: Deep Learning Model Compression Exacerbates Privacy Leakage
Model compression is crucial for minimizing memory storage and accelerating inference in deep learning (DL) models, including recent foundation models like large language models (LLMs). Users can access different compres…
Model CompressionOptimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach
The use of HSI for autonomous navigation is a promising research field aimed at improving the accuracy and robustness of detection, tracking, and scene understanding systems based on vision sensors. Combining advanced co…
Scene UnderstandingModel CompressionLINR-PCGC: Lossless Implicit Neural Representations for Point Cloud Geometry Compression
Existing AI-based point cloud compression methods struggle with dependence on specific training data distributions, which limits their real-world deployment. Implicit Neural Representation (INR) methods solve the above p…
DecoderModel CompressionApplication-Specific Component-Aware Structured Pruning of Deep Neural Networks in Control via Soft Coefficient Optimization
Deep neural networks (DNNs) offer significant flexibility and robust performance. This makes them ideal for building not only system models but also advanced neural network controllers (NNCs). However, their high complex…
Model CompressionGeLaCo: An Evolutionary Approach to Layer Compression
Large Language Models (LLM) have achieved remarkable performance across a large number of tasks, but face critical deployment and usage barriers due to substantial computational requirements. Model compression methods, w…
Model CompressionCompression Method for Deep Diagonal State Space Model Based on $H^2$ Optimal Reduction
Deep learning models incorporating linear SSMs have gained attention for capturing long-range dependencies in sequential data. However, their large parameter sizes pose challenges for deployment on resource-constrained d…
Model CompressionOn Information Geometry and Iterative Optimization in Model Compression: Operator Factorization
The ever-increasing parameter counts of deep learning models necessitate effective compression techniques for deployment on resource-constrained devices. This paper explores the application of information geometry, the s…
Model CompressionExploring the Limits of Model Compression in LLMs: A Knowledge Distillation Study on QA Tasks
Large Language Models (LLMs) have demonstrated outstanding performance across a range of NLP tasks, however, their computational demands hinder their deployment in real-world, resource-constrained environments. This work…
Knowledge DistillationQuestion AnsweringModel CompressionGradients as an Action: Towards Communication-Efficient Federated Recommender Systems via Adaptive Action Sharing
As a promising privacy-aware collaborative model training paradigm, Federated Learning (FL) is becoming popular in the design of distributed recommender systems. However, Federated Recommender Systems (FedRecs) greatly s…
Recommendation SystemsFederated LearningModel CompressionPut Teacher in Student's Shoes: Cross-Distillation for Ultra-compact Model Compression Framework
In the era of mobile computing, deploying efficient Natural Language Processing (NLP) models in resource-restricted edge settings presents significant challenges, particularly in environments requiring strict privacy com…
Natural Language UnderstandingModel CompressionModel Compression using Progressive Channel Pruning
In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods t…
Model CompressionTransfer LearningMGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMs
The enormous parameter scale of large language models (LLMs) has made model compression a research hotspot, which aims to alleviate computational resource demands during deployment and inference. As a promising direction…
Model CompressionMoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE
Mixture-of-Experts (MoE) enables efficient scaling of large language models by activating only a subset of experts per input token. However, deploying MoE-based models incurs significant memory overhead due to the need t…
Model CompressionKronecker-factored Approximate Curvature (KFAC) From Scratch
Kronecker-factored approximate curvature (KFAC) is arguably one of the most prominent curvature approximations in deep learning. Its applications range from optimization to Bayesian deep learning, training data attributi…
Model CompressionDipSVD: Dual-importance Protected SVD for Efficient LLM Compression
The ever-increasing computational demands and deployment costs of large language models (LLMs) have spurred numerous compressing methods. Compared to quantization and unstructured pruning, SVD compression offers superior…
Model CompressionQuantizationRLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models
Vision-Language-Action models (VLA) have demonstrated remarkable capabilities and promising potential in solving complex robotic manipulation tasks. However, their substantial parameter sizes and high inference latency p…
Model CompressionQuantizationVision-Language-ActionModel compression using knowledge distillation with integrated gradients
Model compression is critical for deploying deep learning models on resource-constrained devices. We introduce a novel method enhancing knowledge distillation with integrated gradients (IG) as a data augmentation strateg…
Data AugmentationKnowledge DistillationModel Compression