Stochastic Optimization
12개 벤치마크 · 논문 1,566편 · 이 태스크의 논문 보기 →
Benchmarks
Most implemented
Adam: A Method for Stochastic Optimization
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
SGDR: Stochastic Gradient Descent with Warm Restarts
On the Variance of the Adaptive Learning Rate and Beyond
Lookahead Optimizer: k steps forward, 1 step back
Papers
Convergence rates for the RMSprop optimizer with full control of the hyperparameters
Popular adaptive stochastic gradient descent (SGD) methods to train artificial intelligence (AI) systems include the RMSprop, the Adam, and the AdamW optimizers, where the adaptivity parts in Adam and AdamW basically jus…
Stochastic OptimizationBeyond Optimal Rates in Stochastic Optimization: Trajectory-Adaptive Stopping Rules
Stochastic gradient descent (SGD) is typically analyzed at a deterministic horizon chosen before the algorithm is run, even though practical stopping decisions are made adaptively by inspecting the evolving trajectory. T…
Stochastic OptimizationStochastic Saddle Avoidance Beyond Unit Excitation and Smoothness: A Pathwise Lyapunov-Perron Framework
Unit excitation (UE) is a common assumption in stochastic saddle avoidance: the stochastic error must have a uniformly positive component along every direction, in expectation. This condition gives a direct way to rule o…
Stochastic OptimizationQuantum Speedups for Stochastic Optimization with Heavy-Tailed Noise
We study stochastic optimization with heavy-tailed gradient noise. We first propose a novel quantum mean estimator for multivariate heavy-tailed random variables that achieves lower query complexity than optimal classica…
Stochastic OptimizationEnhanced Neural Quantum State via Annealed Gradient Descent
Neural quantum states offer expressive representations of quantum many-body wave functions, yet their practical accuracy can be limited by stochastic optimization rather than representational capacity. Here we identify a…
Stochastic OptimizationTamed Stochastic Gradient Hamiltonian Monte Carlo
In this paper, we propose a novel tamed stochastic gradient Hamiltonian Monte Carlo (tSGHMC) algorithm for sampling and stochastic optimization problems with superlinearly growing stochastic gradients. Under a certain co…
Stochastic Optimization