Sparse Autoencoder

2000년 도입 · 논문 57편에서 사용

A Sparse Autoencoder is a type of autoencoder that employs sparsity to achieve an information bottleneck. Specifically the loss function is constructed so that activations are penalized within a layer. The sparsity constraint can be imposed with L1 regularization or a KL divergence between expected average neuron activation to an ideal distribution $p$. Image: Jeff Jordan. Read his blog post (click) for a detailed summary of autoencoders.