DHEL

2000년 도입 · 논문 1편에서 사용

InfoNCE variants demonstrate direct and indirect coupling between the alignment and uniformity terms thus hurting optimisation. The Decoupled Hyperspherical Energy Loss (DHEL) is an NT-Xent variant that completly decouples alignment from uniformity by discarding the corresponding terms from the denominator. In this way optimisation is more efficient and robust to hyper parameter changes.

출처: Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses

소개 논문: Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses