Progressive Automatic Design of Search Space for One-Shot Neural Architecture Search
Neural Architecture Search (NAS) has attracted growing interest. To reduce the search cost, recent work has explored weight sharing across models and made major progress in One-Shot NAS. However, it has been observed that a model with higher one-shot model accuracy does not necessarily perform better when stand-alone trained. To address this issue, in this paper, we propose Progressive Automatic Design of search space, named PAD-NAS. Unlike previous approaches where the same operation search space is shared by all the layers in the supernet, we formulate a progressive search strategy based on operation pruning and build a layer-wise operation search space. In this way, PAD-NAS can automatically design the operations for each layer and achieve a trade-off between search space quality and model diversity. During the search, we also take the hardware platform constraints into consideration for efficient neural network model deployment. Extensive experiments on ImageNet show that our method can achieve state-of-the-art performance.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityEfficient Neural NetworkNeural Architecture SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System
Recently, deep learning models have been widely spread in the industrial recommender systems and boosted the recommendation quality. Though having achieved remarkable success, the design of task-aware recommender systems…
Feature EngineeringNeural Architecture SearchRecommendation SystemsHalf Search Space is All You Need
Neural Architecture Search (NAS) is a powerful tool for automating architecture design. One-Shot NAS techniques, such as DARTS, have gained substantial popularity due to their combination of search efficiency with simpli…
AllGPUNeural Architecture SearchProgressive Feature Interaction Search for Deep Sparse Network
Deep sparse networks (DSNs), of which the crux is exploring the high-order feature interactions, have become the state-of-the-art on the prediction task with high-sparsity features. However, these models suffer from low …
Neural Architecture SearchPONAS: Progressive One-shot Neural Architecture Search for Very Efficient Deployment
We achieve very efficient deep learning model deployment that designs neural network architectures to fit different hardware constraints. Given a constraint, most neural architecture search (NAS) methods either sample a …
Neural Architecture SearchHEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning
One-shot methods have significantly advanced the field of neural architecture search (NAS) by adopting weight-sharing strategy to reduce search costs. However, the accuracy of performance estimation can be compromised by…
Neural Architecture Search