Sparsemax

2000년 도입 · 논문 27편에서 사용

Sparsemax is a type of activation/output function similar to the traditional softmax, but able to output sparse probabilities. $$ \text{sparsemax}\left(z\right) = \arg\_{p∈\Delta^{K−1}}\min||\mathbf{p} - \mathbf{z}||^{2} $$

출처: From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification

소개 논문: From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification