Hardtanh Activation

2000년 도입

Hardtanh is an activation function used for neural networks: $$ f\left(x\right) = -1 \text{ if } x < - 1 $$ $$ f\left(x\right) = x \text{ if } -1 \leq x \leq 1 $$ $$ f\left(x\right) = 1 \text{ if } x > 1 $$ It is a cheaper and more computationally efficient version of the tanh activation. Image Source: Zhuan Lan