Neural Architecture Search in Embedding Space
The neural architecture search (NAS) algorithm with reinforcement learning can be a powerful and novel framework for the automatic discovering process of neural architectures. However, its application is restricted by noncontinuous and high-dimensional search spaces, which result in difficulty in optimization. To resolve these problems, we proposed NAS in embedding space (NASES), which is a novel framework. Unlike other NAS with reinforcement learning approaches that search over a discrete and high-dimensional architecture space, this approach enables reinforcement learning to search in an embedding space by using architecture encoders and decoders. The current experiment demonstrated that the performance of the final architecture network using the NASES procedure is comparable with that of other popular NAS approaches for the image classification task on CIFAR-10. The results of the experiment were efficient and indicated that NASES was highly efficient to discover final architecture only in $<$3.5 GPU hours. The beneficial-performance and effectiveness of NASES was impressive when the architecture-embedding searching and weight initialization were applied.
Code (0)
등록된 구현이 없습니다.
Tasks
GPUimage-classificationImage ClassificationNeural Architecture Searchreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contrastive Embeddings for Neural Architectures
The performance of algorithms for neural architecture search strongly depends on the parametrization of the search space. We use contrastive learning to identify networks across different initializations based on their d…
Contrastive LearningNeural Architecture SearchTransfer LearningLearnable Embedding Space for Efficient Neural Architecture Compression
We propose a method to incrementally learn an embedding space over the domain of network architectures, to enable the careful selection of architectures for evaluation during compressed architecture search. Given a teach…
Bayesian OptimizationNeural Architecture SearchReinforcement LearningSmooth Variational Graph Embeddings for Efficient Neural Architecture Search
Neural architecture search (NAS) has recently been addressed from various directions, including discrete, sampling-based methods and efficient differentiable approaches. While the former are notoriously expensive, the la…
AutoMLBayesian OptimizationGraph Neural NetworkNeural Architecture SearchTowards Code-Oriented LM Embeddings for Surrogate-Assisted Neural Architecture Search
Developing effective surrogates (performance predictors) for Neural Architecture Search (NAS) typically requires expensive fine-tuning or the engineering of complex representations. We propose a low-cost embedding strate…
Neural Architecture SearchNASGEM: Neural Architecture Search via Graph Embedding Method
Neural Architecture Search (NAS) automates and prospers the design of neural networks. Estimator-based NAS has been proposed recently to model the relationship between architectures and their performance to enable scalab…
Graph EmbeddingGraph SimilarityNeural Architecture Searchobject-detection+1