paper-with-me

홈 › Papers

Regularized Evolution for Image Classifier Architecture Search

2018-02-05 · Esteban Real, Alok Aggarwal, Yanping Huang, Quoc V. Le

The effort devoted to hand-crafting neural network image classifiers has motivated the use of architecture search to discover them automatically. Although evolutionary algorithms have been repeatedly applied to neural network topologies, the image classifiers thus discovered have remained inferior to human-crafted ones. Here, we evolve an image classifier---AmoebaNet-A---that surpasses hand-designs for the first time. To do this, we modify the tournament selection evolutionary algorithm by introducing an age property to favor the younger genotypes. Matching size, AmoebaNet-A has comparable accuracy to current state-of-the-art ImageNet models discovered with more complex architecture-search methods. Scaled to larger size, AmoebaNet-A sets a new state-of-the-art 83.9% / 96.6% top-5 ImageNet accuracy. In a controlled comparison against a well known reinforcement learning algorithm, we give evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search. This is relevant when fewer compute resources are available. Evolution is, thus, a simple method to effectively discover high-quality architectures.

📄 PDF Abstract BibTeX arXiv:1802.01548

Code (5)

DataCanvasIO/Hypernets tf
anonDEHB/DEHB
tally0818/NASNet pytorch
tensorflow/tpu/tree/master/models/official/amoeba_net tf
xuanhungho/nsga pytorch

Tasks

Evolutionary AlgorithmsImage ClassificationNeural Architecture SearchReinforcement Learning

Methods 이 논문이 사용한 방법론

Aging Evolution Aging Evolution, or Regularized Evolution, is an evolutionary algorithm for neural architecture search.…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
RMSProp RMSProp is an unpublished adaptive learning rate optimizer proposed by Geoff Hinton. The motivation…
ScheduledDropPath 설명 없음
Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…
SGD with Momentum 설명 없음
Weight Decay 설명 없음

Similar Papers 제목 키워드 기반

Regularized Evolutionary Algorithm for Dynamic Neural Topology Search

2019-05-15 · Cristiano Saltori, Subhankar Roy, Nicu Sebe, Giovanni Iacca

Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures…

Evolutionary AlgorithmsNeural Architecture SearchObject Recognition

Neural Loss Function Evolution for Large-Scale Image Classifier Convolutional Neural Networks

2024-01-30 · Brandon Morgan, Dean Hougen

For classification, neural networks typically learn by minimizing cross-entropy, but are evaluated and compared using accuracy. This disparity suggests neural loss function search (NLFS), the search for a drop-in replace…

Image Augmentation

Understanding Patterns of Deep Learning ModelEvolution in Network Architecture Search

2023-09-22 · Robert Underwood, Meghana Madhastha, Randal Burns, Bogdan Nicolae

Network Architecture Search and specifically Regularized Evolution is a common way to refine the structure of a deep learning model.However, little is known about how models empirically evolve over time which has design …

Deep LearningScheduling

Differential Evolution for Neural Architecture Search

2020-12-11 · Noor Awad, Neeratyoy Mallik, Frank Hutter

Neural architecture search (NAS) methods rely on a search strategy for deciding which architectures to evaluate next and a performance estimation strategy for assessing their performance (e.g., using full evaluations, mu…

Bayesian OptimizationNeural Architecture Search

Neural Predictor for Neural Architecture Search

2019-12-02 · ECCV 2020 8 · Wei Wen, Hanxiao Liu, Hai Li, Yiran Chen 외

Neural Architecture Search methods are effective but often use complex algorithms to come up with the best architecture. We propose an approach with three basic steps that is conceptually much simpler. First we train N r…

Neural Architecture Searchregression