InPlace-ABN

2000년 도입 · 논문 2편에서 사용

In-Place Activated Batch Normalization, or InPlace-ABN, substitutes the conventionally used succession of BatchNorm + Activation layers with a single plugin layer, hence avoiding invasive framework surgery while providing straightforward applicability for existing deep learning frameworks. It approximately halves the memory requirements during training of modern deep learning models.

출처: In-Place Activated BatchNorm for Memory-Optimized Training of DNNs

소개 논문: In-Place Activated BatchNorm for Memory-Optimized Training of DNNs