EmProx: Neural Network Performance Estimation For Neural Architecture Search
Common Neural Architecture Search methods generate large amounts of candidate architectures that need training in order to assess their performance and find an optimal architecture. To minimize the search time we use different performance estimation strategies. The effectiveness of such strategies varies in terms of accuracy and fit and query time. This study proposes a new method, EmProx Score (Embedding Proximity Score). Similar to Neural Architecture Optimization (NAO), this method maps candidate architectures to a continuous embedding space using an encoder-decoder framework. The performance of candidates is then estimated using weighted kNN based on the embedding vectors of architectures of which the performance is known. Performance estimations of this method are comparable to the MLP performance predictor used in NAO in terms of accuracy, while being nearly nine times faster to train compared to NAO. Benchmarking against other performance estimation strategies currently used shows similar to better accuracy, while being five up to eighty times faster.
Code (1)
Tasks
BenchmarkingDecoderNeural Architecture SearchSimilar Papers 제목 키워드 기반
Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention
Sequential recommender systems identify user preferences from their past interactions to predict subsequent items optimally. Although traditional deep-learning-based models and modern transformer-based models in previous…
Contrastive LearningRecommendation SystemsSequential RecommendationLightweight Monocular Depth with a Novel Neural Architecture Search Method
This paper presents a novel neural architecture search method, called LiDNAS, for generating lightweight monocular depth estimation models. Unlike previous neural architecture search (NAS) approaches, where finding optim…
Depth EstimationDiversityMonocular Depth EstimationNeural Architecture SearchEPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search
Neural Architecture Search (NAS) has shown excellent results in designing architectures for computer vision problems. NAS alleviates the need for human-defined settings by automating architecture design and engineering. …
GPUNeural Architecture SearchRethinking Performance Estimation in Neural Architecture Search
Neural architecture search (NAS) remains a challenging problem, which is attributed to the indispensable and time-consuming component of performance estimation (PE). In this paper, we provide a novel yet systematic rethi…
Neural Architecture SearchZero-Shot Neural Architecture Search with Weighted Response Correlation
Neural architecture search (NAS) is a promising approach for automatically designing neural network architectures. However, the architecture estimation of NAS is computationally expensive and time-consuming because of tr…
Neural Architecture Search