Deep Active Learning with a Neural Architecture Search
We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby the learning algorithm searches for effective architectures on the fly, while actively learning. We apply our strategy using three known querying techniques (softmax response, MC-dropout, and coresets) and show that the proposed approach overwhelmingly outperforms active learning using fixed architectures.
Code (1)
Tasks
Active LearningNeural Architecture SearchSimilar Papers 제목 키워드 기반
IDARTS: Interactive Differentiable Architecture Search
Differentiable Architecture Search (DARTS) improves the efficiency of architecture search by learning the architecture and network parameters end-to-end. However, the intrinsic relationship between the architecture's…
ALBench: A Framework for Evaluating Active Learning in Object Detection
Active learning is an important technology for automated machine learning systems. In contrast to Neural Architecture Search (NAS) which aims at automating neural network architecture design, active learning aims at auto…
Active Learningimage-classificationImage ClassificationNeural Architecture Search+3Open-Domain Conversational Search Assistant with Transformers
Open-domain conversational search assistants aim at answering user questions about open topics in a conversational manner. In this paper we show how the Transformer architecture achieves state-of-the-art results in key I…
Conversational SearchRetrievalSearch Space Adaptation for Differentiable Neural Architecture Search in Image Classification
As deep neural networks achieve unprecedented performance in various tasks, neural architecture search (NAS), a research field for designing neural network architectures with automated processes, is actively underway. Mo…
image-classificationImage ClassificationNeural Architecture SearchSummarizing long regulatory documents with a multi-step pipeline
Due to their length and complexity, long regulatory texts are challenging to summarize. To address this, a multi-step extractive-abstractive architecture is proposed to handle lengthy regulatory documents more effectivel…
Decoder