BAGUA

2000년 도입 · 논문 1편에서 사용

BAGUA is a communication framework whose design goal is to provide a system abstraction that is both flexible and modular to support state-of-the-art system relaxation techniques of distributed training. The abstraction goes beyond parameter server and Allreduce paradigms, and provides a collection of MPI-style collective operations to facilitate communications with different precision and centralization strategies.

출처: BAGUA: Scaling up Distributed Learning with System Relaxations

소개 논문: BAGUA: Scaling up Distributed Learning with System Relaxations