EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
Neural Architecture Search (NAS) has shown excellent results in designing architectures for computer vision problems. NAS alleviates the need for human-defined settings by automating architecture design and engineering. However, NAS methods tend to be slow, as they require large amounts of GPU computation. This bottleneck is mainly due to the performance estimation strategy, which requires the evaluation of the generated architectures, mainly by training them, to update the sampler method. In this paper, we propose EPE-NAS, an efficient performance estimation strategy, that mitigates the problem of evaluating networks, by scoring untrained networks and creating a correlation with their trained performance. We perform this process by looking at intra and inter-class correlations of an untrained network. We show that EPE-NAS can produce a robust correlation and that by incorporating it into a simple random sampling strategy, we are able to search for competitive networks, without requiring any training, in a matter of seconds using a single GPU. Moreover, EPE-NAS is agnostic to the search method, since it focuses on the evaluation of untrained networks, making it easy to integrate into almost any NAS method.
Code (2)
Tasks
GPUNeural Architecture SearchSimilar Papers 제목 키워드 기반
MONAS: Efficient Zero-Shot Neural Architecture Search for MCUs
Neural Architecture Search (NAS) has proven effective in discovering new Convolutional Neural Network (CNN) architectures, particularly for scenarios with well-defined accuracy optimization goals. However, previous appro…
Edge-computingNeural Architecture SearchPEng4NN: An Accurate Performance Estimation Engine for Efficient Automated Neural Network Architecture Search
Neural network (NN) models are increasingly used in scientific simulations, AI, and other high performance computing (HPC) fields to extract knowledge from datasets. Each dataset requires tailored NN model architecture, …
Neural Architecture SearchEfficient Multi-Objective Neural Architecture Search via Pareto Dominance-based Novelty Search
Neural Architecture Search (NAS) aims to automate the discovery of high-performing deep neural network architectures. Traditional objective-based NAS approaches typically optimize a certain performance metric (e.g., pred…
DiversityNeural Architecture SearchReNAS: Relativistic Evaluation of Neural Architecture Search
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS). To save computational cost, most of existing NAS algorithms often train and eval…
Neural Architecture SearchReNAS:Relativistic Evaluation of Neural Architecture Search
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS). To save computational cost, most of existing NAS algorithms often train and evaluate…
Neural Architecture Search