Conditional Accelerated Lazy Stochastic Gradient Descent

2017-03-16 · ICML 2017 8 · Guanghui Lan, Sebastian Pokutta, Yi Zhou, Daniel Zink

In this work we introduce a conditional accelerated lazy stochastic gradient descent algorithm with optimal number of calls to a stochastic first-order oracle and convergence rate $O\left(\frac{1}{\varepsilon^2}\right)$ improving over the projection-free, Online Frank-Wolfe based stochastic gradient descent of Hazan and Kale [2012] with convergence rate $O\left(\frac{1}{\varepsilon^4}\right)$.

📄 PDF Abstract BibTeX arXiv:1703.05840

Code (0)

등록된 구현이 없습니다.

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds

2025-02-20 · Conglong Xu, Peiqi Yang, Hao Wu

We solve a regularized weighted low-rank approximation problem by a stochastic gradient descent on a manifold. To guarantee the convergence of our stochastic gradient descent, we establish a convergence theorem on manifo…

Optimal Adaptive and Accelerated Stochastic Gradient Descent

2018-10-01 · Qi Deng, Yi Cheng, Guanghui Lan

Stochastic gradient descent (\textsc{Sgd}) methods are the most powerful optimization tools in training machine learning and deep learning models. Moreover, acceleration (a.k.a. momentum) methods and diagonal scaling (a.…

BIG-bench Machine Learning Stochastic Optimization

regression Stochastic Optimization

Conditional Accelerated Lazy Stochastic Gradient Descent

Code (0)

Similar Papers 제목 키워드 기반

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Accelerated Almost-Sure Convergence Rates for Nonconvex Stochastic Gradient Descent using Stochastic Learning Rates

Stochastic Proximal Gradient Descent with Acceleration Techniques

Accelerating Stochastic Gradient Descent For Least Squares Regression