paper-with-me

홈 › Papers

Linear Convergence of Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

2015-12-31 · Bo Wen, Xiaojun Chen, Ting Kei Pong

In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a threshold such that if the extrapolation coefficients are chosen below this threshold, then the sequence generated converges $R$-linearly to a stationary point of the problem. Moreover, the corresponding sequence of objective values is also $R$-linearly convergent. In addition, the threshold reduces to $1$ for convex problems and, as a consequence, we obtain the $R$-linear convergence of the sequence generated by FISTA with fixed restart. Finally, we present some numerical experiments to illustrate our results.

📄 PDF Abstract BibTeX arXiv:1512.09302

Code (1)

EvanZhuang/MRI-Reconstruction-with-Sparse-Optimization

Similar Papers 제목 키워드 기반

An Accelerated Block Proximal Framework with Adaptive Momentum for Nonconvex and Nonsmooth Optimization

2023-08-23 · Weifeng Yang, Wenwen Min

We propose an accelerated block proximal linear framework with adaptive momentum (ABPL$^+$) for nonconvex and nonsmooth optimization. We analyze the potential causes of the extrapolation step failing in some algorithms, …

Tensor Decomposition

Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization

2017-05-14 · ICML 2017 8 · Qunwei Li, Yi Zhou, Yingbin Liang, Pramod K. Varshney

In many modern machine learning applications, structures of underlying mathematical models often yield nonconvex optimization problems. Due to the intractability of nonconvexity, there is a rising need to develop efficie…

Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems

2017-11-18 · Lei Yang

In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To …

Variable Selection

Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexity

2025-01-25 · Shuailong Zhu, Xiaohui Chen

The proximal algorithm is a powerful tool to minimize nonlinear and nonsmooth functionals in a general metric space. Motivated by the recent progress in studying the training dynamics of the noisy gradient descent algori…

Tighter Performance Theory of FedExProx

2024-10-20 · Wojciech Anyszka, Kaja Gruntkowska, Alexander Tyurin, Peter Richtárik

We revisit FedExProx - a recently proposed distributed optimization method designed to enhance convergence properties of parallel proximal algorithms via extrapolation. In the process, we uncover a surprising flaw: its k…

Distributed OptimizationDiversityFederated Learning