paper-with-me

홈 › Papers

The Incremental Proximal Method: A Probabilistic Perspective

2018-07-12 · Ömer Deniz Akyildiz, Victor Elvira, Joaquin Miguez

In this work, we highlight a connection between the incremental proximal method and stochastic filters. We begin by showing that the proximal operators coincide, and hence can be realized with, Bayes updates. We give the explicit form of the updates for the linear regression problem and show that there is a one-to-one correspondence between the proximal operator of the least-squares regression and the Bayes update when the prior and the likelihood are Gaussian. We then carry out this observation to a general sequential setting: We consider the incremental proximal method, which is an algorithm for large-scale optimization, and show that, for a linear-quadratic cost function, it can naturally be realized by the Kalman filter. We then discuss the implications of this idea for nonlinear optimization problems where proximal operators are in general not realizable. In such settings, we argue that the extended Kalman filter can provide a systematic way for the derivation of practical procedures.

📄 PDF Abstract BibTeX arXiv:1807.04594

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Methods 이 논문이 사용한 방법론

Linear Regression Linear Regression is a method for modelling a relationship between a dependent variable and independent variables. These models can be fit with numerous approaches. The most…

Similar Papers 제목 키워드 기반

A probabilistic incremental proximal gradient method

2018-12-04 · Ömer Deniz Akyildiz, Émilie Chouzenoux, Víctor Elvira, Joaquín Míguez

In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm.…

Last Iterate Convergence of Incremental Methods and Applications in Continual Learning

2024-03-11 · Xufeng Cai, Jelena Diakonikolas

Incremental gradient and incremental proximal methods are a fundamental class of optimization algorithms used for solving finite sum problems, broadly studied in the literature. Yet, without strong convexity, their conve…

Continual Learning

General Proximal Incremental Aggregated Gradient Algorithms: Better and Novel Results under General Scheme

2019-10-11 · NeurIPS 2019 12 · Tao Sun, Yuejiao Sun, Dongsheng Li, Qing Liao

The incremental aggregated gradient algorithm is popular in network optimization and machine learning research. However, the current convergence results require the objective function to be strongly convex. And the exist…

SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization

2022-07-17 · Pourya Behmandpoor, Puya Latafat, Andreas Themelis, Marc Moonen 외

We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. Each iteration of SPIRAL consists of an inner …

Scalable nonconvex inexact proximal splitting

2012-12-01 · NeurIPS 2012 12 · Suvrit Sra

We study large-scale, nonsmooth, nonconconvex optimization problems. In particular, we focus on nonconvex problems with \emph{composite} objectives. This class of problems includes the extensively studied convex, composi…