Finite Sample and Large Deviations Analysis of Stochastic Gradient Algorithm with Correlated Noise
We analyze the finite sample regret of a decreasing step size stochastic gradient algorithm. We assume correlated noise and use a perturbed Lyapunov function as a systematic approach for the analysis. Finally we analyze the escape time of the iterates using large deviations theory.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Gaussian stochastic volatility models: Scaling regimes, large deviations, and moment explosions
In this paper, we establish sample path large and moderate deviation principles for log-price processes in Gaussian stochastic volatility models, and study the asymptotic behavior of exit probabilities, call pricing func…
Using random perturbations to infer the structure of feedback control in gene expression
Feedback in cellular processes is typically inferred through cellular responses to experimental perturbations. Modular response analysis provides a theoretical framework for translating specific perturbations into feedba…
A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
Building on the large-sample analysis of infinitesimal gradient boosting (Dombry and Duchamps, 2024b), we study the fluctuations of the process around its deterministic limit and establish a functional central limit theo…
Data Smashing 2.0: Sequence Likelihood (SL) Divergence For Fast Time Series Comparison
Recognizing subtle historical patterns is central to modeling and forecasting problems in time series analysis. Here we introduce and develop a new approach to quantify deviations in the underlying hidden generators of o…
Feature EngineeringTime SeriesTime Series AnalysisStochastic Differential Equations models for Least-Squares Stochastic Gradient Descent
We study the dynamics of a continuous-time model of the Stochastic Gradient Descent (SGD) for the least-square problem. Indeed, pursuing the work of Li et al. (2019), we analyze Stochastic Differential Equations (SDEs) t…