Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses by Sompolinsky and Zippelius (1982), it has since been used to derive asymptotic dynamical equations for a wide range of models in physics, high-dimensional statistics and machine learning. One of the main tools used by physicists to obtain these equations is the dynamical cavity method, which has remained largely non-rigorous. In contrast, existing mathematical formalizations have relied on alternative approaches, including Gaussian conditioning, large deviations over paths, or Fourier analysis. In this work, we formalize the dynamical cavity method and use it to give a new proof of the DMFT equations for General First Order Methods, a broad class of dynamics encompassing algorithms such as Gradient Descent and Approximate Message Passing.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Rigorous dynamical mean field theory for stochastic gradient descent methods
We prove closed-form equations for the exact high-dimensional asymptotics of a family of first order gradient-based methods, learning an estimator (e.g. M-estimator, shallow neural network, ...) from observations on Gaus…
Small-time asymptotics for Gaussian self-similar stochastic volatility models
We consider the class of self-similar Gaussian stochastic volatility models, and compute the small-time (near-maturity) asymptotics for the corresponding asset price density, the call and put pricing functions, and the i…
ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm
We study the problem of solving strongly convex and smooth unconstrained optimization problems using stochastic first-order algorithms. We devise a novel algorithm, referred to as Recursive One-Over-T SGD (ROOT-SGD), bas…
Stochastic OptimizationUnityFirst-Order Asymptotics of Path-Dependent Derivatives in Multiscale Stochastic Volatility Environment
In this paper, we extend the first-order asymptotics analysis of Fouque et al. to general path-dependent financial derivatives using Dupire's functional Ito calculus. The main conclusion is that the market group paramete…
Short Maturity Asian Options in Local Volatility Models
We present a rigorous study of the short maturity asymptotics for Asian options with continuous-time averaging, under the assumption that the underlying asset follows a local volatility model. The asymptotics for out-of-…