Learning Diverse-Structured Networks for Adversarial Robustness
In adversarial training (AT), the main focus has been the objective and optimizer while the model has been less studied, so that the models being used are still those classic ones in standard training (ST). Classic network architectures (NAs) are generally worse than searched NAs in ST, which should be the same in AT. In this paper, we argue that NA and AT cannot be handled independently, since given a dataset, the optimal NA in ST would be no longer optimal in AT. That being said, AT is time-consuming itself; if we directly search NAs in AT over large search spaces, the computation will be practically infeasible. Thus, we propose a diverse-structured network (DS-Net), to significantly reduce the size of the search space: instead of low-level operations, we only consider predefined atomic blocks, where an atomic block is a time-tested building block like the residual block. There are only a few atomic blocks and thus we can weight all atomic blocks rather than find the best one in a searched block of DS-Net, which is an essential trade-off between exploring diverse structures and exploiting the best structures. Empirical results demonstrate the advantages of DS-Net, i.e., weighting the atomic blocks.
Code (1)
Tasks
Adversarial RobustnessSimilar Papers 제목 키워드 기반
How to Build Robust FAQ Chatbot with Controllable Question Generator?
Many unanswerable adversarial questions fool the question-answer (QA) system with some plausible answers. Building a robust, frequently asked questions (FAQ) chatbot needs a large amount of diverse adversarial examples. …
ChatbotDiversityPassage RetrievalQuestion Generation+3ExAL: An Exploration Enhanced Adversarial Learning Algorithm
Adversarial learning is critical for enhancing model robustness, aiming to defend against adversarial attacks that jeopardize machine learning systems. Traditional methods often lack efficient mechanisms to explore diver…
On the Adversarial Robustness of Graph Contrastive Learning Methods
Contrastive learning (CL) has emerged as a powerful framework for learning representations of images and text in a self-supervised manner while enhancing model robustness against adversarial attacks. More recently, resea…
Adversarial RobustnessContrastive LearningGraph ClassificationNode ClassificationCSTAR: Towards Compact and STructured Deep Neural Networks with Adversarial Robustness
Model compression and model defense for deep neural networks (DNNs) have been extensively and individually studied. Considering the co-importance of model compactness and robustness in practical applications, several pri…
Adversarial RobustnessModel CompressionSupervised Adversarial Contrastive Learning for Emotion Recognition in Conversations
Extracting generalized and robust representations is a major challenge in emotion recognition in conversations (ERC). To address this, we propose a supervised adversarial contrastive learning (SACL) framework for learnin…
Contrastive LearningEmotion RecognitionEmotion Recognition in Conversation