Self-Adjusting Smooth L1 Loss

2000년 도입 · 논문 1편에서 사용

Self-Adjusting Smooth L1 Loss is a loss function used in object detection that was introduced with RetinaMask. This is an improved version of Smooth L1. For Smooth L1 loss we have: $$ f(x) = 0.5 \frac{x^{2}}{\beta} \text{ if } |x| < \beta $$ $$ f(x) = |x| -0.5\beta \text{ otherwise } $$ Here a point $\beta$ splits the positive axis range into two parts: $L2$ loss is used for targets in range $[0, \beta]$, and $L1$ loss is used beyond $\beta$ to avoid over-penalizing utliers. The overall function is smooth (continuous, together with its derivative). However, the choice of control point ($\beta$) is heuristic and is usually done by hyper parameter search. Instead, with self-adjusting smooth L1 loss, inside the loss function the running mean and variance of the absolute loss are recorded. We use the running minibatch mean and variance with a momentum of $0.9$ to update these two parameters.

출처: RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free

소개 논문: RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free