PipeMare

2000년 도입 · 논문 1편에서 사용

PipeMare is an asynchronous (bubble-free) pipeline parallel method for training large neural networks. It involves two main techniques: learning rate rescheduling and discrepancy correction.

출처: PipeMare: Asynchronous Pipeline Parallel DNN Training

소개 논문: PipeMare: Asynchronous Pipeline Parallel DNN Training