Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamics
We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a Markov chain, merely a \emph{Markov chain in a random environment}, which complicates the mathematical treatment considerably. We derive a strong law of large numbers and a functional central limit theorem for SGLD.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Quantitative Functional Central Limit Theorem for Shallow Neural Networks
We prove a Quantitative Functional Central Limit Theorem for one-hidden-layer neural networks with generic activation function. The rates of convergence that we establish depend heavily on the smoothness of the activatio…
Affine Jump-Diffusions: Stochastic Stability and Limit Theorems
Affine jump-diffusions constitute a large class of continuous-time stochastic models that are particularly popular in finance and economics due to their analytical tractability. Methods for parameter estimation for such …
parameter estimationFunctional Central Limit Theorem for Stochastic Gradient Descent
We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the …
Central Limit Theorems for Asynchronous Averaged Q-Learning
This paper establishes central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates. We prove a non-asymptotic central limit theorem, where the convergence rate in Wasserstein distance explici…
A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
Building on the large-sample analysis of infinitesimal gradient boosting (Dombry and Duchamps, 2024b), we study the fluctuations of the process around its deterministic limit and establish a functional central limit theo…