StarReLU

2000년 도입 · 논문 1편에서 사용

$s \cdot (\mathrm{ReLU}(x))^2 + b$ where $s \in \mathbb{R}$ and $b \in \mathbb{R}$ are shared for all channels and can be set as constants (s=0.8944, b=-0.4472) or learnable parameters.

출처: MetaFormer Baselines for Vision

소개 논문: MetaFormer Baselines for Vision