L1 Regularization

1986년 도입 · 논문 90편에서 사용

$L_{1}$ Regularization is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L\_{1}$ Norm of the weights: $$L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{||w||}\_{1}$$ where $\lambda$ is a value determining the strength of the penalty. In contrast to weight decay, $L_{1}$ regularization promotes sparsity; i.e. some parameters have an optimal value of zero. Image Source: Wikipedia#/media/File:Sparsityl1.png)