BatchChannel Normalization

2000년 도입 · 논문 1편에서 사용

Batch-Channel Normalization, or BCN, uses batch knowledge to prevent channel-normalized models from getting too close to "elimination singularities". Elimination singularities correspond to the points on the training trajectory where neurons become consistently deactivated. They cause degenerate manifolds in the loss landscape which will slow down training and harm model performances.

출처: Rethinking Normalization and Elimination Singularity in Neural Networks

소개 논문: Rethinking Normalization and Elimination Singularity in Neural Networks