Effective Approaches to Batch Parallelization for Dynamic Neural Network Architectures
We present a simple dynamic batching approach applicable to a large class of dynamic architectures that consistently yields speedups of over 10x. We provide performance bounds when the architecture is not known a priori and a stronger bound in the special case where the architecture is a predetermined balanced tree. We evaluate our approach on Johnson et al.'s recent visual question answering (VQA) result of his CLEVR dataset by Inferring and Executing Programs (IEP). We also evaluate on sparsely gated mixture of experts layers and achieve speedups of up to 1000x over the naive implementation.
Code (1)
Tasks
Mixture-of-ExpertsQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Seesaw: High-throughput LLM Inference via Model Re-sharding
To improve the efficiency of distributed large language model (LLM) inference, various parallelization strategies, such as tensor and pipeline parallelism, have been proposed. However, the distinct computational characte…
Computational EfficiencyLanguage ModelingLanguage ModellingLarge Language Model+1GANGR: GAN-Assisted Scalable and Efficient Global Routing Parallelization
Global routing is a critical stage in electronic design automation (EDA) that enables early estimation and optimization of the routability of modern integrated circuits with respect to congestion, power dissipation, and …
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
Accelerating stochastic gradient methods with classical momentum schemes, such as Polyak's heavy ball, has proven highly successful in training large-scale machine learning models, particularly when combined with the har…
Minibatch and Parallelization for Online Large Margin Structured Learning
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning
In this paper we aim to formally explain the phenomenon of fast convergence of SGD observed in modern machine learning. The key observation is that most modern learning architectures are over-parametrized and are trained…