AdaHessian

2000년 도입 · 논문 6편에서 사용

ADAHESSIAN is a new stochastic optimization algorithm that directly incorporates approximate curvature information from the loss function, and it includes several novel performance-improving features, including a fast Hutchinson based method to approximate the curvature matrix with low computational overhead.

출처: ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

소개 논문: ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning