Sophia
Second-order Clipped Stochastic Optimization
2000년 도입 · 논문 2편에서 사용
Please enter a description about the method here
출처: Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
소개 논문: Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Optimization · General