Sparse Switchable Normalization

2000년 도입 · 논문 1편에서 사용

Sparse Switchable Normalization (SSN) is a variant on Switchable Normalization where the importance ratios are constrained to be sparse. Unlike $\ell_1$ and $\ell_0$ constraints that impose difficulties in optimization, the constrained optimization problem is turned into feed-forward computation through SparseMax, which is a sparse version of softmax.

출처: SSN: Learning Sparse Switchable Normalization via SparsestMax

소개 논문: SSN: Learning Sparse Switchable Normalization via SparsestMax