Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation
We propose NeuronaBox, a flexible, user-friendly, and high-fidelity approach to emulate DNN training workloads. We argue that to accurately observe performance, it is possible to execute the training workload on a subset of real nodes and emulate the networked execution environment along with the collective communication operations. Initial results from a proof-of-concept implementation show that NeuronaBox replicates the behavior of actual systems with high accuracy, with an error margin of less than 1% between the emulated measurements and the real system.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Latent Representation Learning Framework for Hyperspectral Image Emulation in Remote Sensing
Synthetic hyperspectral image (HSI) generation is essential for large-scale simulation, algorithm development, and mission design, yet traditional radiative transfer models remain computationally expensive and proposed e…
Representation LearningA Distributed Emulation Environment for In-Memory Computing Systems
In-memory computing technology is used extensively in artificial intelligence devices due to lower power consumption and fast calculation of matrix-based functions. The development of such a device and its integration in…
Real-time Digital RF Emulation -- II: A Near Memory Custom Accelerator
A near memory hardware accelerator, based on a novel direct path computational model, for real-time emulation of radio frequency systems is demonstrated. Our evaluation of hardware performance uses both application-speci…
On the Value of Tokeniser Pretraining in Physics Foundation Models
We investigate the impact of tokeniser pretraining on the accuracy and efficiency of physics emulation. Modern high-resolution simulations produce vast volumes of data spanning diverse physical regimes and scales. Traini…
Computational EfficiencyEmulation of physical processes with Emukit
Decision making in uncertain scenarios is an ubiquitous challenge in real world systems. Tools to deal with this challenge include simulations to gather information and statistical emulation to quantify uncertainty. The …
Bayesian OptimizationDecision MakingDecision Making Under UncertaintyExperimental Design