Reducing Neural Architecture Search Spaces with Training-Free Statistics and Computational Graph Clustering
The computational demands of neural architecture search (NAS) algorithms are usually directly proportional to the size of their target search spaces. Thus, limiting the search to high-quality subsets can greatly reduce the computational load of NAS algorithms. In this paper, we present Clustering-Based REDuction (C-BRED), a new technique to reduce the size of NAS search spaces. C-BRED reduces a NAS space by clustering the computational graphs associated with its architectures and selecting the most promising cluster using proxy statistics correlated with network accuracy. When considering the NAS-Bench-201 (NB201) data set and the CIFAR-100 task, C-BRED selects a subset with 70% average accuracy instead of the whole space's 64% average accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringGraph ClusteringNeural Architecture SearchSimilar Papers 제목 키워드 기반
Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights
Deep learning has revolutionized computer vision, but it achieved its tremendous success using deep network architectures which are mostly hand-crafted and therefore likely suboptimal. Neural Architecture Search (NAS…
image-classificationImage ClassificationNeural Architecture SearchTraining-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights
Deep learning has revolutionized computer vision, but it achieved its tremendous success using deep network architectures which are mostly hand-crafted and therefore likely suboptimal. Neural Architecture Search (NAS) ai…
image-classificationImage ClassificationNeural Architecture SearchTransferrable Surrogates in Expressive Neural Architecture Search Spaces
Neural architecture search (NAS) faces a challenge in balancing the exploration of expressive, broad search spaces that enable architectural innovation with the need for efficient evaluation of architectures to effective…
Neural Architecture SearchConstruction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars
The discovery of neural architectures from simple building blocks is a long-standing goal of Neural Architecture Search (NAS). Hierarchical search spaces are a promising step towards this goal but lack a unifying search …
AutoMLBayesian OptimizationNeural Architecture SearchEvolutionary Neural Architecture Search for 3D Point Cloud Analysis
Neural architecture search (NAS) automates neural network design by using optimization algorithms to navigate architecture spaces, reducing the burden of manual architecture design. While NAS has achieved success, applyi…
Evolutionary AlgorithmsNavigateNeural Architecture Search