paper-with-me

Papers

An Accelerated Proximal Coordinate Gradient Method

2014-12-01 · NeurIPS 2014 12 · Qihang Lin, Zhaosong Lu, Lin Xiao

We develop an accelerated randomized proximal coordinate gradient (APCG) method, for solving a broad class of composite convex optimization problems. In particular, our method achieves faster linear convergence rates for minimizing strongly convex functions than existing randomized proximal coordinate gradient methods. We show how to apply the APCG method to solve the dual of the regularized empirical risk minimization (ERM) problem, and devise efficient implementations that can avoid full-dimensional vector operations. For ill-conditioned ERM problems, our method obtains improved convergence rates than the state-of-the-art stochastic dual coordinate ascent (SDCA) method.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Anderson acceleration of coordinate descent

2020-11-19 · Quentin Bertrand, Mathurin Massias

Acceleration of first order methods is mainly obtained via inertial techniques \`a la Nesterov, or via nonlinear extrapolation. The latter has known a recent surge of interest, with successful applications to gradient an…

regression

Accelerated Block Coordinate Proximal Gradients with Applications in High Dimensional Statistics

2017-10-15 · Tsz Kit Lau, Yuan YAO

Nonconvex optimization problems arise in different research fields and arouse lots of attention in signal processing, statistics and machine learning. In this work, we explore the accelerated proximal gradient method and…

BIG-bench Machine LearningregressionVocal Bursts Intensity Prediction

The Physical Systems Behind Optimization Algorithms

2016-12-08 · NeurIPS 2018 12 · Lin F. Yang, R. Arora, V. Braverman, Tuo Zhao

We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent,…

BIG-bench Machine Learning

Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification

2024-10-12 · Kexin Li, Luwei Bai, Xiao Wang, Hao Wang

Anderson acceleration is an effective technique for enhancing the efficiency of fixed-point iterations; however, analyzing its convergence in nonsmooth settings presents significant challenges. In this paper, we investig…

Stochastic Proximal Gradient Descent with Acceleration Techniques

2014-12-01 · NeurIPS 2014 12 · Atsushi Nitanda

Proximal gradient descent (PGD) and stochastic proximal gradient descent (SPGD) are popular methods for solving regularized risk minimization problems in machine learning and statistics. In this paper, we propose and ana…