FastFusionNet: New State-of-the-Art for DAWNBench SQuAD
In this technical report, we introduce FastFusionNet, an efficient variant of FusionNet [12]. FusionNet is a high performing reading comprehension architecture, which was designed primarily for maximum retrieval accuracy with less regard towards computational requirements. For FastFusionNets we remove the expensive CoVe layers [21] and substitute the BiLSTMs with far more efficient SRU layers [19]. The resulting architecture obtains state-of-the-art results on DAWNBench [5] while achieving the lowest training and inference time on SQuAD [25] to-date. The code is available at https://github.com/felixgwu/FastFusionNet.
Code (2)
Tasks
Reading ComprehensionRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasin…
BenchmarkingBIG-bench Machine LearningGPUSuper-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
In this paper, we describe a phenomenon, which we named "super-convergence", where neural networks can be trained an order of magnitude faster than with standard training methods. The existence of super-convergence is re…
Know What You Don't Know: Unanswerable Questions for SQuAD
Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in t…
Natural Language UnderstandingQuestion AnsweringReading ComprehensionThe Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models
Pretrained language models have achieved super-human performances on many Machine Reading Comprehension (MRC) benchmarks. Nevertheless, their relative inability to defend against adversarial attacks has spurred skepticis…
Adversarial AttackMachine Reading ComprehensionNatural Language UnderstandingReading ComprehensionEQuANt (Enhanced Question Answer Network)
Machine Reading Comprehension (MRC) is an important topic in the domain of automated question answering and in natural language processing more generally. Since the release of the SQuAD 1.1 and SQuAD 2 datasets, progress…
Machine Reading ComprehensionMulti-Task LearningQuestion AnsweringReading Comprehension