ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hours
AlphaFold2 has been hailed as a breakthrough in protein folding. It can rapidly predict protein structures with lab-grade accuracy. However, its implementation does not include the necessary training code. OpenFold is the first trainable public reimplementation of AlphaFold. AlphaFold training procedure is prohibitively time-consuming, and gets diminishing benefits from scaling to more compute resources. In this work, we conducted a comprehensive analysis on the AlphaFold training procedure based on Openfold, identified that inefficient communications and overhead-dominated computations were the key factors that prevented the AlphaFold training from effective scaling. We introduced ScaleFold, a systematic training method that incorporated optimizations specifically for these factors. ScaleFold successfully scaled the AlphaFold training to 2080 NVIDIA H100 GPUs with high resource utilization. In the MLPerf HPC v3.0 benchmark, ScaleFold finished the OpenFold benchmark in 7.51 minutes, shown over $6\times$ speedup than the baseline. For training the AlphaFold model from scratch, ScaleFold completed the pretraining in 10 hours, a significant improvement over the seven days required by the original AlphaFold pretraining baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
Protein FoldingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle
Accurate protein structure prediction can significantly accelerate the development of life science. The accuracy of AlphaFold2, a frontier end-to-end structure prediction system, is already close to that of the experimen…
Protein Structure PredictionFastFold: Reducing AlphaFold Training Time from 11 Days to 67 Hours
Protein structure prediction helps to understand gene translation and protein function, which is of growing interest and importance in structural biology. The AlphaFold model, which used transformer architecture to achie…
Protein Structure PredictionTranslationTechnical Report of HelixFold3 for Biomolecular Structure Prediction
The AlphaFold series has transformed protein structure prediction with remarkable accuracy, often matching experimental methods. AlphaFold2, AlphaFold-Multimer, and the latest AlphaFold3 represent significant strides in …
PredictionProtein Structure PredictionAlphaFold two years on: validation and impact
Two years on from the initial release of AlphaFold2 we have seen its widespread adoption as a structure prediction tool. Here we discuss some of the latest work based on AlphaFold2, with a particular focus on its use wit…
APACE: AlphaFold2 and advanced computing as a service for accelerated discovery in biophysics
The prediction of protein 3D structure from amino acid sequence is a computational grand challenge in biophysics, and plays a key role in robust protein structure prediction algorithms, from drug discovery to genome inte…
Drug DiscoveryPredictionProtein Structure Predictionscientific discovery