ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search Spaces
In this paper, we approach the problem of optimizing blackbox functions over large hybrid search spaces consisting of both combinatorial and continuous parameters. We demonstrate that previous evolutionary algorithms which rely on mutation-based approaches, while flexible over combinatorial spaces, suffer from a curse of dimensionality in high dimensional continuous spaces both theoretically and empirically, which thus limits their scope over hybrid search spaces as well. In order to combat this curse, we propose ES-ENAS, a simple and modular joint optimization procedure combining the class of sample-efficient smoothed gradient techniques, commonly known as Evolutionary Strategies (ES), with combinatorial optimizers in a highly scalable and intuitive way, inspired by the one-shot or supernet paradigm introduced in Efficient Neural Architecture Search (ENAS). By doing so, we achieve significantly more sample efficiency, which we empirically demonstrate over synthetic benchmarks, and are further able to apply ES-ENAS for architecture search over popular RL benchmarks.
Code (2)
Tasks
Combinatorial OptimizationContinuous ControlEvolutionary AlgorithmsMultiobjective OptimizationNeural Architecture SearchQuantizationSimilar Papers 제목 키워드 기반
NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search
Neural architecture search (NAS) is a promising method for automatically design neural architectures. NAS adopts a search strategy to explore the predefined search space to find outstanding performance architecture with …
Bayesian OptimizationEvolutionary AlgorithmsNeural Architecture SearchSampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search
The performance of a deep neural network is heavily dependent on its architecture and various neural architecture search strategies have been developed for automated network architecture design. Recently, evolutionary ne…
Evolutionary Algorithmsglobal-optimizationNeural Architecture SearchBenchENAS: A Benchmarking Platform for Evolutionary Neural Architecture Search
Neural architecture search (NAS), which automatically designs the architectures of deep neural networks, has achieved breakthrough success over many applications in the past few years. Among different classes of NAS meth…
BenchmarkingGPUNeural Architecture SearchBenchENAS: A Benchmarking Platform for Evolutionary Neural Architecture Search
Neural architecture search (NAS), which automatically designs the architectures of deep neural networks, has achieved breakthrough success over many applications in the past few years. Among different classes of NAS meth…
BenchmarkingGPUNeural Architecture SearchEvolutionary Neural Architecture Search with Dual Contrastive Learning
Evolutionary Neural Architecture Search (ENAS) has gained attention for automatically designing neural network architectures. Recent studies use a neural predictor to guide the process, but the high computational costs o…
Neural Architecture SearchSelf-Supervised LearningContrastive Learning