Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each intermediate layer and reduce the number of neurons from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as {ASNet}, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98$\times$ parameter and 7.30$\times$ flops reduction. The universal active subspace attack vector can achieve around 20% higher attack ratio compared with the existing approach in all of our numerical experiments. The PyTorch codes for this paper are available online.
Code (1)
Tasks
Adversarial AttackUncertainty QuantificationSimilar Papers 제목 키워드 기반
Surrogate assisted active subspace and active subspace assisted surrogate -- A new paradigm for high dimensional structural reliability analysis
Performing reliability analysis on complex systems is often computationally expensive. In particular, when dealing with systems having high input dimensionality, reliability estimation becomes a daunting task. A popular …
Sparse LearningDecoupling Semantics and Fingerprints: A Universal Representation for AI-Generated Image Detection
Detecting AI-generated images across unseen architectures remains challenging, as existing models often overfit to generator-specific fingerprints and semantic content rather than learning universal forgery traces. We at…
Data-driven Meets Geometric Control: Zero Dynamics, Subspace Stabilization, and Malicious Attacks
Studying structural properties of linear dynamical systems through invariant subspaces is one of the key contributions of the geometric approach to system theory. In general, a model of the dynamics is required in order …
FLARE: Toward Universal Dataset Purification against Backdoor Attacks
Deep neural networks (DNNs) are susceptible to backdoor attacks, where adversaries poison datasets with adversary-specified triggers to implant hidden backdoors, enabling malicious manipulation of model predictions. Data…
AllFingerprinting Deep Neural Networks Globally via Universal Adversarial Perturbations
In this paper, we propose a novel and practical mechanism which enables the service provider to verify whether a suspect model is stolen from the victim model via model extraction attacks. Our key insight is that the pro…
Contrastive LearningModel extraction