SiLU

2000년 도입 · 논문 30편에서 사용

Sigmoid Linear Units, or SiLUs, are activation functions for neural networks. The activation of the SiLU is computed by the sigmoid function multiplied by its input, or $$ x\sigma(x).$$ See Gaussian Error Linear Units (GELUs) where the SiLU was originally coined, and see Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning and Swish: a Self-Gated Activation Function where the SiLU was experimented with later.

출처: Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning

소개 논문: Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning