BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels
One-Shot methods have evolved into one of the most popular methods in Neural Architecture Search (NAS) due to weight sharing and single training of a supernet. However, existing methods generally suffer from two issues: predetermined number of channels in each layer which is suboptimal; and model averaging effects and poor ranking correlation caused by weight coupling and continuously expanding search space. To explicitly address these issues, in this paper, a Broadening-and-Shrinking One-Shot NAS (BS-NAS) framework is proposed, in which broadening' refers to broadening the search space with a spring block enabling search for numbers of channels during training of the supernet; while shrinking' refers to a novel shrinking strategy gradually turning off those underperforming operations. The above innovations broaden the search space for wider representation and then shrink it by gradually removing underperforming operations, followed by an evolutionary algorithm to efficiently search for the optimal architecture. Extensive experiments on ImageNet illustrate the effectiveness of the proposed BS-NAS as well as the state-of-the-art performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Neural Architecture SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AutoPruning for Deep Neural Network with Dynamic Channel Masking
Modern deep neural network models are large and computationally intensive. One typical solution to this issue is model pruning. However, most current pruning algorithms depend on hand crafted rules or domain expertise. T…
AutoMLNetwork PruningESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by the broadening interests of the spoken language translation community. ESPnet-ST-v2 supports 1) offline speech-to-text translation (ST), 2) si…
BenchmarkingSimultaneous Speech-to-Text TranslationSpeech-to-Speech TranslationSpeech-to-Text+2On- Device Information Extraction from Screenshots in form of tags
We propose a method to make mobile screenshots easily searchable. In this paper, we present the workflow in which we: 1) preprocessed a collection of screenshots, 2) identified script presentin image, 3) extracted unstru…
FormTAGPartial Channel Dependence with Channel Masks for Time Series Foundation Models
Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale TS datasets. However, previous efforts have primarily focused on design…
Anomaly DetectionImputationTime SeriesDeep Learning Enabled Real-Time Photoacoustic Tomography System via Single Data Acquisition Channel
Photoacoustic computed tomography (PACT) combines the optical contrast of optical imaging and the penetrability of sonography. In this work, we develop a novel PACT system to provide real-time imaging, which is achieved …