Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff
As a widely adopted model compression technique, model pruning has demonstrated strong effectiveness across various architectures. However, we observe that when sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance. This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms, rendering them inoperable. To overcome this limitation, we propose a bidirectional pruning-regrowth strategy. Starting from an extremely compressed network that satisfies hardware constraints, the method selectively regenerates critical connections to recover lost performance, effectively mitigating the sharp accuracy drop commonly observed under high sparsity conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Model CompressionSimilar Papers 제목 키워드 기반
Pruning of Deep Spiking Neural Networks through Gradient Rewiring
Spiking Neural Networks (SNNs) have been attached great importance due to their biological plausibility and high energy-efficiency on neuromorphic chips. As these chips are usually resource-constrained, the compression o…
EntryPrune: Neural Network Feature Selection using First Impressions
There is an ongoing effort to develop feature selection algorithms to improve interpretability, reduce computational resources, and minimize overfitting in predictive models. Neural networks stand out as architectures on…
feature selectionComprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks
Graph Neural Networks (GNNs) tend to suffer from high computation costs due to the exponentially increasing scale of graph data and the number of model parameters, which restricts their utility in practical applications.…
Node ClassificationCA-AFP: Cluster-Aware Adaptive Federated Pruning
Federated Learning (FL) faces major challenges in real-world deployments due to statistical heterogeneity across clients and system heterogeneity arising from resource-constrained devices. While clustering-based approach…
Human Activity RecognitionFederated LearningA procedure for automated tree pruning suggestion using LiDAR scans of fruit trees
In fruit tree growth, pruning is an important management practice for preventing overcrowding, improving canopy access to light and promoting regrowth. Due to the slow nature of agriculture, decisions in pruning are typi…
Decision MakingManagement