paper-with-me

홈 › Papers

FGP: Feature-Gradient-Prune for Efficient Convolutional Layer Pruning

2024-11-19 · Qingsong Lv, Jiasheng Sun, Sheng Zhou, Xu Zhang, Liangcheng Li, Yun Gao, Sun Qiao, Jie Song, Jiajun Bu

To reduce computational overhead while maintaining model performance, model pruning techniques have been proposed. Among these, structured pruning, which removes entire convolutional channels or layers, significantly enhances computational efficiency and is compatible with hardware acceleration. However, existing pruning methods that rely solely on image features or gradients often result in the retention of redundant channels, negatively impacting inference efficiency. To address this issue, this paper introduces a novel pruning method called Feature-Gradient Pruning (FGP). This approach integrates both feature-based and gradient-based information to more effectively evaluate the importance of channels across various target classes, enabling a more accurate identification of channels that are critical to model performance. Experimental results demonstrate that the proposed method improves both model compactness and practicality while maintaining stable performance. Experiments conducted across multiple tasks and datasets show that FGP significantly reduces computational costs and minimizes accuracy loss compared to existing methods, highlighting its effectiveness in optimizing pruning outcomes. The source code is available at: https://github.com/FGP-code/FGP.

📄 PDF Abstract BibTeX arXiv:2411.12781

Code (1)

fgp-code/fgp 공식 구현 pytorch

Tasks

Computational Efficiency

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Similar Papers 제목 키워드 기반

Compact Deep Convolutional Neural Networks With Coarse Pruning

2016-10-30 · Sajid Anwar, Wonyong Sung

The learning capability of a neural network improves with increasing depth at higher computational costs. Wider layers with dense kernel connectivity patterns furhter increase this cost and may hinder real-time inference…

Network Pruning

Reconstructing Pruned Filters using Cheap Spatial Transformations

2021-10-25 · Roy Miles, Krystian Mikolajczyk

We present an efficient alternative to the convolutional layer using cheap spatial transformations. This construction exploits an inherent spatial redundancy of the learned convolutional filters to enable a much greater …

Feature CompressionKnowledge DistillationModel Compression

Runtime Neural Pruning

2017-12-01 · NeurIPS 2017 12 · Ji Lin, Yongming Rao, Jiwen Lu, Jie zhou

In this paper, we propose a Runtime Neural Pruning (RNP) framework which prunes the deep neural network dynamically at the runtime. Unlike existing neural pruning methods which produce a fixed pruned model for deployment…

Reinforcement Learning

Iterative Filter Pruning for Concatenation-based CNN Architectures

2024-05-04 · Svetlana Pavlitska, Oliver Bagge, Federico Peccia, Toghrul Mammadov 외

Model compression and hardware acceleration are essential for the resource-efficient deployment of deep neural networks. Modern object detectors have highly interconnected convolutional layers with concatenations. In thi…

Model Compression

GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs

2026-01-27 · Wei Huang, Anda Cheng, Yinggui Wang arxiv

Fine-tuning Large Language Models (LLMs) with downstream data is often considered time-consuming and expensive. Structured pruning methods are primarily employed to improve the inference efficiency of pre-trained models.…

Knowledge Distillation