Adversarial Robustness vs. Model Compression, or Both?
It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training can provide a notion of security against adversarial attacks. However, adversarial robustness requires a significantly larger capacity of the network than that for the natural training with only benign examples. This paper proposes a framework of concurrent adversarial training and weight pruning that enables model compression while still preserving the adversarial robustness and essentially tackles the dilemma of adversarial training. Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting; training a small model from scratch even with inherited initialization from the large model cannot achieve neither adversarial robustness nor high standard accuracy. Code is available at https://github.com/yeshaokai/Robustness-Aware-Pruning-ADMM.
Code (1)
Tasks
Adversarial RobustnessmodelModel CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Benchmarking Adversarial Robustness of Compressed Deep Learning Models
The increasing size of Deep Neural Networks (DNNs) poses a pressing need for model compression, particularly when employed on resource constrained devices. Concurrently, the susceptibility of DNNs to adversarial attacks …
Adversarial RobustnessBenchmarkingDeep LearningModel CompressionAdversarial Robustness vs Model Compression, or Both?
It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training …
Adversarial RobustnessmodelModel CompressionNetwork PruningHolistic Adversarially Robust Pruning
Neural networks can be drastically shrunk in size by removing redundant parameters. While crucial for the deployment on resource-constraint hardware, oftentimes, compression comes with a severe drop in accuracy and lack …
Adversarial RobustnessDynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introd…
Adversarial RobustnessLow-rank compressionRelationship between Model Compression and Adversarial Robustness: A Review of Current Evidence
Increasing the model capacity is a known approach to enhance the adversarial robustness of deep learning networks. On the other hand, various model compression techniques, including pruning and quantization, can reduce t…
Adversarial RobustnessModel CompressionQuantization