Fraternal Dropout

2000년 도입 · 논문 2편에서 사용

Fraternal Dropout is a regularization method for recurrent neural networks that trains two identical copies of an RNN (that share parameters) with different dropout masks while minimizing the difference between their (pre-softmax) predictions. This encourages the representations of RNNs to be invariant to dropout mask, thus being robust.