FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
Recent DNN pruning algorithms have succeeded in reducing the number of parameters in fully connected layers, often with little or no drop in classification accuracy. However, most of the existing pruning schemes either have to be applied during training or require a costly retraining procedure after pruning to regain classification accuracy. We start by proposing a cheap pruning algorithm for fully connected DNN layers based on difference of convex functions (DC) optimisation, that requires little or no retraining. We then provide a theoretical analysis for the growth in the Generalization Error (GE) of a DNN for the case of bounded perturbations to the hidden layers, of which weight pruning is a special case. Our pruning method is orders of magnitude faster than competing approaches, while our theoretical analysis sheds light to previously observed problems in DNN pruning. Experiments on commnon feedforward neural networks validate our results.
Code (1)
Tasks
General ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FetalDiffusion: Pose-Controllable 3D Fetal MRI Synthesis with Conditional Diffusion Model
The quality of fetal MRI is significantly affected by unpredictable and substantial fetal motion, leading to the introduction of artifacts even when fast acquisition sequences are employed. The development of 3D real-tim…
GPUPose EstimationGeneralization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketching
In this paper, we derive a novel bound on the generalization error of Magnitude-Based pruning of overparameterized neural networks. Our work builds on the bounds in Arora et al. [2018] where the error depends on one, the…
Generalization BoundsEnhancing the Regularization Effect of Weight Pruning in Artificial Neural Networks
Artificial neural networks (ANNs) may not be worth their computational/memory costs when used in mobile phones or embedded devices. Parameter-pruning algorithms combat these costs, with some algorithms capable of removin…
General Classificationimage-classificationImage ClassificationClassification of fetal compromise during labour: signal processing and feature engineering of the cardiotocograph
Cardiotocography (CTG) is the main tool used for fetal monitoring during labour. Interpretation of CTG requires dynamic pattern recognition in real time. It is recognised as a difficult task with high inter- and intra-ob…
BIG-bench Machine LearningFeature EngineeringFetal Pose Estimation in Volumetric MRI using a 3D Convolution Neural Network
The performance and diagnostic utility of magnetic resonance imaging (MRI) in pregnancy is fundamentally constrained by fetal motion. Motion of the fetus, which is unpredictable and rapid on the scale of conventional ima…
DiagnosticPose EstimationTime SeriesTime Series Analysis