Hierarchical Representations for Efficient Architecture Search

2017-11-01 · ICLR 2018 1 · Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu

We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches. We also present results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36 hours down to 1 hour.

📄 PDF Abstract BibTeX arXiv:1711.00436

Code (1)

sjoshi804/hierarchical-darts-neural-architecture-search pytorch

Tasks

General Classification image-classification Image Classification Neural Architecture Search

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Learning Hierarchical Relational Representations through Relational Convolutions

2023-10-05 · Awni Altabaa, John Lafferty

An evolving area of research in deep learning is the study of architectures and inductive biases that support the learning of relational feature representations. In this paper, we address the challenge of learning repres…

Relation

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

2021-08-02 · Zachary Ravichandran, Lisa Peng, Nathan Hughes, J. Daniel Griffith 외

Representations are crucial for a robot to learn effective navigation policies. Recent work has shown that mid-level perceptual abstractions, such as depth estimates or 2D semantic segmentation, lead to more effective po…

2D Semantic Segmentation Graph Neural Network Semantic Segmentation

Genre classification Music Genre Classification Neural Architecture Search

Hierarchical Representations for Efficient Architecture Search

Code (1)

Tasks

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반

Learning Hierarchical Relational Representations through Relational Convolutions

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

Unsupervised inference for optimizing deep feedforward neural network architecture

Learning representations in Bayesian Confidence Propagation neural networks

Hierarchical quantum circuit representations for neural architecture search