k-Sparse Autoencoder

2000년 도입 · 논문 3편에서 사용

k-Sparse Autoencoders are autoencoders with linear activation function, where in hidden layers only the $k$ highest activities are kept. This achieves exact sparsity in the hidden representation. Backpropagation only goes through the the top $k$ activated units. This can be achieved with a ReLU layer with an adjustable threshold.

출처: k-Sparse Autoencoders

소개 논문: k-Sparse Autoencoders