PMish

2000년 도입 · 논문 1편에서 사용

The Parametric Mish, or PMish, is a modification of the Mish activation function that has a trainable parameter $\beta$. This $\beta$ parameter allows the network to dynamically adjust the smoothness of the activation function transition. \begin{equation} f(x) = x \cdot \tanh\left(\frac{\ln(1 + e^{\beta x})}{\beta}\right), \beta > 0, \end{equation}

출처: Enhancing GANs with MMD Neural Architecture Search, PMish Activation Function, and Adaptive Rank Decomposition

소개 논문: Enhancing GANs with MMD Neural Architecture Search, PMish Activation Function, and Adaptive Rank Decomposition