paper-with-me

Papers

RL-Pruner: Structured Pruning Using Reinforcement Learning for CNN Compression and Acceleration

2024-11-10 · Boyao Wang, Volodymyr Kindratenko

Convolutional Neural Networks (CNNs) have demonstrated exceptional performance in recent years. Compressing these models not only reduces storage requirements, making deployment to edge devices feasible, but also accelerates inference, thereby reducing latency and computational costs. Structured pruning, which removes filters at the layer level, directly modifies the model architecture. This approach achieves a more compact architecture while maintaining target accuracy, ensuring that the compressed model retains good compatibility and hardware efficiency. Our method is based on a key observation: filters in different layers of a neural network have varying importance to the model's performance. When the number of filters to prune is fixed, the optimal pruning distribution across different layers is uneven to minimize performance loss. Layers that are more sensitive to pruning should account for a smaller proportion of the pruning distribution. To leverage this insight, we propose RL-Pruner, which uses reinforcement learning to learn the optimal pruning distribution. RL-Pruner can automatically extract dependencies between filters in the input model and perform pruning, without requiring model-specific pruning implementations. We conducted experiments on models such as GoogleNet, ResNet, and MobileNet, comparing our approach to other structured pruning methods to validate its effectiveness. Our code is available at https://github.com/Beryex/RLPruner-CNN.

📄 PDF Abstract BibTeX arXiv:2411.06463

Code (1)

beryex/rlpruner-cnn 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Kaiming Initialization 설명 없음
Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…
Pruning 설명 없음
Average Pooling 설명 없음
Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…
Global Average Pooling Global Average Pooling is a pooling operation designed to replace fully connected layers in classical CNNs. The idea is to generate one feature map for each corresponding…

Similar Papers 제목 키워드 기반

TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models

2022-03-30 · ACL 2022 5 · Ziqing Yang, Yiming Cui, Zhigang Chen

Pre-trained language models have been prevailed in natural language processing and become the backbones of many NLP tasks, but the demands for computational resources have limited their applications. In this paper, we in…

Model Compression

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models

2024-08-07 · Pengxiang Zhao, Hanyu Hu, Ping Li, Yi Zheng 외

Pruning is a critical strategy for compressing trained large language models (LLMs), aiming at substantial memory conservation and computational acceleration without compromising performance. However, existing pruning me…

Adapt-Pruner: Adaptive Structural Pruning for Efficient Small Language Model Training

2025-02-05 · Boyao Wang, Rui Pan, Shizhe Diao, Xingyuan Pan 외

Small language models (SLMs) have attracted considerable attention from both academia and industry due to their broad range of applications in edge devices. To obtain SLMs with strong performance, conventional approaches…

Language ModelingLanguage ModellingMMLUSmall Language Model

Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimization

2026-08-13 · Palaash Goel, Ayan Sengupta, Akshay Nambi, Tanmoy Chakraborty arxiv

Structured pruning is a promising approach for compressing large language models (LLMs), yet existing methods rely heavily on greedy heuristics that produce myopic decisions, and often fail to precisely meet target compr…

FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Difference

2025-08-04 · Chenqing Lin, Mostafa Hussien, Chengyao Yu, Bingyi Jing 외 arxiv

Structured pruning is a standard tool for compressing deep neural networks, but its practical performance depends on how sparsity is allocated across layers. We propose FAIR-Pruner, a search-free framework for adaptive l…