Adaptive Input Representations

2000년 도입 · 논문 66편에서 사용

Adaptive Input Embeddings extend the adaptive softmax to input word representations. The factorization assigns more capacity to frequent words and reduces the capacity for less frequent words with the benefit of reducing overfitting to rare words.

출처: Adaptive Input Representations for Neural Language Modeling

소개 논문: Adaptive Input Representations for Neural Language Modeling