Neural Architecture Generator Optimization
Neural Architecture Search (NAS) was first proposed to achieve state-of-the-art performance through the discovery of new architecture patterns, without human intervention. An over-reliance on expert knowledge in the search space design has however led to increased performance (local optima) without significant architectural breakthroughs, thus preventing truly novel solutions from being reached. In this work we 1) are the first to investigate casting NAS as a problem of finding the optimal network generator and 2) we propose a new, hierarchical and graph-based search space capable of representing an extremely large variety of network types, yet only requiring few continuous hyper-parameters. This greatly reduces the dimensionality of the problem, enabling the effective use of Bayesian Optimisation as a search strategy. At the same time, we expand the range of valid architectures, motivating a multi-objective learning approach. We demonstrate the effectiveness of this strategy on six benchmark datasets and show that our search space generates extremely lightweight yet highly competitive models.
Code (2)
Tasks
Bayesian OptimisationNeural Architecture SearchvalidSimilar Papers 제목 키워드 기반
Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures
We study the problem of alleviating the instability issue in the GAN training procedure via new architecture design. The discrepancy between the minimax and maximin objective values could serve as a proxy for the difficu…
LEMMADEGAS: Differentiable Efficient Generator Search
Network architecture search (NAS) achieves state-of-the-art results in various tasks such as classification and semantic segmentation. Recently, a reinforcement learning-based approach has been proposed for Generative Ad…
Image GenerationNeural Architecture SearchReinforcement LearningA Hierarchical Architecture for Optimal Unit Commitment and Control of an Ensemble of Steam Generators
A hierarchical architecture for the optimal management of an ensemble of steam generators is presented. The subsystems are coordinated by a multilayer scheme for jointly sustaining a common load. The high level optimizes…
ManagementModel Predictive ControlOracle-Budgeted Molecular Optimization with Short-Term Graph Memory
Molecular optimization is commonly performed under a limited oracle budget, which makes deciding what to evaluate as important as deciding what to generate. We introduce short-term graph memory, a plug-in module that pre…
Wasserstein GAN Can Perform PCA
Generative Adversarial Networks (GANs) have become a powerful framework to learn generative models that arise across a wide variety of domains. While there has been a recent surge in the development of numerous GAN archi…