Prior-Guided One-shot Neural Architecture Search
Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the performance between stand-alone architectures and shared-weight networks. In this paper, we present Prior-Guided One-shot NAS (PGONAS) to strengthen the ranking correlation of supernets. Specifically, we first explore the effect of activation functions and propose a balanced sampling strategy based on the Sandwich Rule to alleviate weight coupling in the supernet. Then, FLOPs and Zen-Score are adopted to guide the training of supernet with ranking correlation loss. Our PGONAS ranks 3rd place in the supernet Track Track of CVPR2022 Second lightweight NAS challenge. Code is available in https://github.com/pprp/CVPR2022-NAS?competition-Track1-3th-solution.
Code (1)
Tasks
Neural Architecture SearchSimilar Papers 제목 키워드 기반
Neural Inheritance Relation Guided One-Shot Layer Assignment Search
Layer assignment is seldom picked out as an independent research topic in neural architecture search. In this paper, for the first time, we systematically investigate the impact of different layer assignments to the netw…
Neural Architecture SearchRelationPosterior-Guided Neural Architecture Search
The emergence of neural architecture search (NAS) has greatly advanced the research on network design. Recent proposals such as gradient-based methods or one-shot approaches significantly boost the efficiency of NAS. In …
GPUimage-classificationImage ClassificationNeural Architecture SearchAGA: Attribute Guided Augmentation
We consider the problem of data augmentation, i.e., generating artificial samples to extend a given corpus of training data. Specifically, we propose attributed-guided augmentation (AGA) which learns a mapping that allow…
AttributeData AugmentationDecoderObject Recognition+2AGA: Attribute-Guided Augmentation
We consider the problem of data augmentation, i.e., generating artificial samples to extend a given corpus of training data. Specifically, we propose attributed-guided augmentation (AGA) which learns a mapping that allow…
AttributeData AugmentationDecoderObject Recognition+2Schema-Guided Paradigm for Zero-Shot Dialog
Developing mechanisms that flexibly adapt dialog systems to unseen tasks and domains is a major challenge in dialog research. Neural models implicitly memorize task-specific dialog policies from the training data. We pos…
MemorizationTransfer Learning