NASGEM: Neural Architecture Search via Graph Embedding Method
Neural Architecture Search (NAS) automates and prospers the design of neural networks. Estimator-based NAS has been proposed recently to model the relationship between architectures and their performance to enable scalable and flexible search. However, existing estimator-based methods encode the architecture into a latent space without considering graph similarity. Ignoring graph similarity in node-based search space may induce a large inconsistency between similar graphs and their distance in the continuous encoding space, leading to inaccurate encoding representation and/or reduced representation capacity that can yield sub-optimal search results. To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method. NASGEM is driven by a novel graph embedding method equipped with similarity measures to capture the graph topology information. By precisely estimating the graph distance and using an auxiliary Weisfeiler-Lehman kernel to guide the encoding, NASGEM can utilize additional structural information to get more accurate graph representation to improve the search efficiency. GEMNet, a set of networks discovered by NASGEM, consistently outperforms networks crafted by existing search methods in classification tasks, i.e., with 0.4%-3.6% higher accuracy while having 11%- 21% fewer Multiply-Accumulates. We further transfer GEMNet for COCO object detection. In both one-stage and twostage detectors, our GEMNet surpasses its manually-crafted and automatically-searched counterparts.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph EmbeddingGraph SimilarityNeural Architecture Searchobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
Smooth Variational Graph Embeddings for Efficient Neural Architecture Search
Neural architecture search (NAS) has recently been addressed from various directions, including discrete, sampling-based methods and efficient differentiable approaches. While the former are notoriously expensive, the la…
AutoMLBayesian OptimizationGraph Neural NetworkNeural Architecture SearchGraph Neural Architecture Search with GPT-4
Graph Neural Architecture Search (GNAS) has shown promising results in automatically designing graph neural networks. However, GNAS still requires intensive human labor with rich domain knowledge to design the search spa…
Neural Architecture SearchArch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search
Neural Architecture Search (NAS) aims to find efficient models for multiple tasks. Beyond seeking solutions for a single task, there are surging interests in transferring network design knowledge across multiple tasks. I…
Neural Architecture SearchRelationInterstellar: Searching Recurrent Architecture for Knowledge Graph Embedding
Knowledge graph (KG) embedding is well-known in learning representations of KGs. Many models have been proposed to learn the interactions between entities and relations of the triplets. However, long-term information amo…
Graph EmbeddingKnowledge Graph EmbeddingNeural Architecture SearchBrain Graph Super-Resolution Using Adversarial Graph Neural Network with Application to Functional Brain Connectivity
Brain image analysis has advanced substantially in recent years with the proliferation of neuroimaging datasets acquired at different resolutions. While research on brain image super-resolution has undergone a rapid deve…
Graph EmbeddingGraph Neural NetworkImage Super-ResolutionSuper-Resolution