paper-with-me

Papers

A Stochastic Proximal Point Algorithm for Saddle-Point Problems

2019-09-13 · Luo Luo, Cheng Chen, Yu-Jun Li, Guangzeng Xie, Zhihua Zhang

We consider saddle point problems which objective functions are the average of $n$ strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence guarantees. However, these methods have a slow convergence when the condition number of the problem is very large. In this paper, we propose a stochastic proximal point algorithm, which accelerates the variance reduction method SAGA for saddle point problems. Compared with the catalyst framework, our algorithm reduces a logarithmic term of condition number for the iteration complexity. We adopt our algorithm to policy evaluation and the empirical results show that our method is much more efficient than state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:1909.06946

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SAGA SAGA is a method in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the…

Similar Papers 제목 키워드 기반

Escaping strict saddle points of the Moreau envelope in nonsmooth optimization

2021-06-17 · Damek Davis, Mateo Díaz, Dmitriy Drusvyatskiy

Recent work has shown that stochastically perturbed gradient methods can efficiently escape strict saddle points of smooth functions. We extend this body of work to nonsmooth optimization, by analyzing an inexact analogu…

Stochastic Gradient Methods with Compressed Communication for Decentralized Saddle Point Problems

2022-05-28 · Chhavi Sharma, Vishnu Narayanan, P. Balamurugan

We develop two compression based stochastic gradient algorithms to solve a class of non-smooth strongly convex-strongly concave saddle-point problems in a decentralized setting (without a central server). Our first algor…

A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach

2019-01-24 · Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil

In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algo…

Escaping Saddle Points for Nonsmooth Weakly Convex Functions via Perturbed Proximal Algorithms

2021-02-04 · Minhui Huang

We propose perturbed proximal algorithms that can provably escape strict saddles for nonsmooth weakly convex functions. The main results are based on a novel characterization of $\epsilon$-approximate local minimum for n…

Optimal Algorithms for Differentially Private Stochastic Monotone Variational Inequalities and Saddle-Point Problems

2021-04-07 · Digvijay Boob, Cristóbal Guzmán

In this work, we conduct the first systematic study of stochastic variational inequality (SVI) and stochastic saddle point (SSP) problems under the constraint of differential privacy (DP). We propose two algorithms: Nois…