EAPruning: Evolutionary Pruning for Vision Transformers and CNNs

2022-10-01 · Qingyuan Li, Bo Zhang, Xiangxiang Chu

Structured pruning greatly eases the deployment of large neural networks in resource-constrained environments. However, current methods either involve strong domain expertise, require extra hyperparameter tuning, or are restricted only to a specific type of network, which prevents pervasive industrial applications. In this paper, we undertake a simple and effective approach that can be easily applied to both vision transformers and convolutional neural networks. Specifically, we consider pruning as an evolution process of sub-network structures that inherit weights through reconstruction techniques. We achieve a 50% FLOPS reduction for ResNet50 and MobileNetV1, leading to 1.37x and 1.34x speedup respectively. For DeiT-Base, we reach nearly 40% FLOPs reduction and 1.4x speedup. Our code will be made available.

📄 PDF Abstract BibTeX arXiv:2210.00181

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Pruning 설명 없음

Pointwise Convolution Pointwise Convolution is a type of convolution that uses a 1x1 kernel: a kernel that iterates through every single point. This…

Depthwise Convolution Depthwise Convolution is a type of convolution where we apply a single convolutional filter for each input channel. In the regular 2D…

Depthwise Separable Convolution While standard convolution performs the channelwise and spatial-wise computation in one step, Depthwise Separable Convolution …

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Average Pooling 설명 없음

Batch Normalization 설명 없음

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…

Isomorphic Pruning for Vision Models

2024-07-05 · Gongfan Fang, Xinyin Ma, Michael Bi Mi, Xinchao Wang

Structured pruning reduces the computational overhead of deep neural networks by removing redundant sub-structures. However, assessing the relative importance of different sub-structures remains a significant challenge, …

SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer

2023-03-30 · CVPR 2023 1 · Xuanyao Chen, Zhijian Liu, Haotian Tang, Li Yi 외

High-resolution images enable neural networks to learn richer visual representations. However, this improved performance comes at the cost of growing computational complexity, hindering their usage in latency-sensitive a…

2D Semantic Segmentation 3D Object Detection Instance Segmentation Monocular 3D Object Detection+4

Elastic ViTs from Pretrained Models without Retraining

2025-10-20 · Walter Simoncini, Michael Dorkenwald, Tijmen Blankevoort, Cees G. M. Snoek 외 arxiv

Vision foundation models achieve remarkable performance but are only available in a limited set of pre-determined sizes, forcing sub-optimal deployment choices under real-world constraints. We introduce SnapViT: Single-s…

On Designing Light-Weight Object Trackers through Network Pruning: Use CNNs or Transformers?

2022-11-24 · Saksham Aggarwal, Taneesh Gupta, Pawan Kumar Sahu, Arnav Chavan 외

Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of s…

Network Pruning Object Object Tracking

Blending Pruning Criteria for Convolutional Neural Networks

2021-07-11 · wei he, Zhongzhan Huang, Mingfu Liang, Senwei Liang 외

The advancement of convolutional neural networks (CNNs) on various vision applications has attracted lots of attention. Yet the majority of CNNs are unable to satisfy the strict requirement for real-world deployment. To …

Clustering Network Pruning