Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling
We demonstrate the first climate-scale, numerical ocean simulations improved through distributed, online inference of Deep Neural Networks (DNN) using SmartSim. SmartSim is a library dedicated to enabling online analysis and Machine Learning (ML) for traditional HPC simulations. In this paper, we detail the SmartSim architecture and provide benchmarks including online inference with a shared ML model on heterogeneous HPC systems. We demonstrate the capability of SmartSim by using it to run a 12-member ensemble of global-scale, high-resolution ocean simulations, each spanning 19 compute nodes, all communicating with the same ML architecture at each simulation timestep. In total, 970 billion inferences are collectively served by running the ensemble for a total of 120 simulated years. Finally, we show our solution is stable over the full duration of the model integrations, and that the inclusion of machine learning has minimal impact on the simulation runtimes.
Code (3)
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Combining Machine Learning with Computational Fluid Dynamics using OpenFOAM and SmartSim
Combining machine learning (ML) with computational fluid dynamics (CFD) opens many possibilities for improving simulations of technical and natural systems. However, CFD+ML algorithms require exchange of data, synchroniz…
In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD
Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O an…
Bridging observation, theory and numerical simulation of the ocean using Machine Learning
Progress within physical oceanography has been concurrent with the increasing sophistication of tools available for its study. The incorporation of machine learning (ML) techniques offers exciting possibilities for advan…
BIG-bench Machine LearningTime Series AnalysisEvaluation of Deep Neural Operator Models toward Ocean Forecasting
Data-driven, deep-learning modeling frameworks have been recently developed for forecasting time series data. Such machine learning models may be useful in multiple domains including the atmospheric and oceanic ones, and…
Time SeriesTowards Fast Simulation of Environmental Fluid Mechanics with Multi-Scale Graph Neural Networks
Numerical simulators are essential tools in the study of natural fluid-systems, but their performance often limits application in practice. Recent machine-learning approaches have demonstrated their ability to accelerate…
Graph Neural Network