Zoneout

2000년 도입 · 논문 29편에서 사용

Zoneout is a method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble, improving generalization. But by preserving instead of dropping hidden units, gradient information and state information are more readily propagated through time, as in feedforward stochastic depth networks.

출처: Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations

소개 논문: Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations