Exponential Decay

2000년 도입 · 논문 110편에서 사용

Exponential Decay is a learning rate schedule where we decay the learning rate with more iterations using an exponential function: $$ \text{lr} = \text{lr}\_{0}\exp\left(-kt\right) $$ Image Credit: Suki Lau