Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS
The algorithms of one-shot neural architecture search(NAS) have been widely used to reduce computation consumption. However, because of the interference among the subnets in which weights are shared, the subnets inherited from these super-net trained by those algorithms have poor consistency in precision ranking. To address this problem, we propose a step-by-step training super-net scheme from one-shot NAS to few-shot NAS. In the training scheme, we firstly train super-net in a one-shot way, and then we disentangle the weights of super-net by splitting them into multi-subnets and training them gradually. Finally, our method ranks 4th place in the CVPR2022 3rd Lightweight NAS Challenge Track1. Our code is available at https://github.com/liujiawei2333/CVPR2022-NAS-competition-Track-1-4th-solution.
Code (1)
Tasks
Neural Architecture SearchSimilar Papers 제목 키워드 기반
Improving Ranking Correlation of Supernet with Candidates Enhancement and Progressive Training
One-shot neural architecture search (NAS) applies weight-sharing supernet to reduce the unaffordable computation overhead of automated architecture designing. However, the weight-sharing technique worsens the ranking con…
Neural Architecture SearchPrior-Guided One-shot Neural Architecture Search
Neural architecture search methods seek optimal candidates with efficient weight-sharing supernet training. However, recent studies indicate poor ranking consistency about the performance between stand-alone architecture…
Neural Architecture SearchSebra: Debiasing Through Self-Guided Bias Ranking
Ranking samples by fine-grained estimates of spuriosity (the degree to which spurious cues are present) has recently been shown to significantly benefit bias mitigation, over the traditional binary biased-\textit{vs}-unb…
Contrastive LearningPi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift
Recently proposed neural architecture search (NAS) methods co-train billions of architectures in a supernet and estimate their potential accuracy using the network weights detached from the supernet. However, the ranking…
AttributeNeural Architecture SearchLearning semantic image quality for fetal ultrasound from noisy ranking annotation
We introduce the notion of semantic image quality for applications where image quality relies on semantic requirements. Working in fetal ultrasound, where ranking is challenging and annotations are noisy, we design a rob…