Local Stochastic Approximation: A Unified View of Federated Learning and Distributed Multi-Task Reinforcement Learning Algorithms

2020-06-24 · Thinh T. Doan

Motivated by broad applications in reinforcement learning and federated learning, we study local stochastic approximation over a network of agents, where their goal is to find the root of an operator composed of the local operators at the agents. Our focus is to characterize the finite-time performance of this method when the data at each agent are generated from Markov processes, and hence they are dependent. In particular, we provide the convergence rates of local stochastic approximation for both constant and time-varying step sizes. Our results show that these rates are within a logarithmic factor of the ones under independent data. We then illustrate the applications of these results to different interesting problems in multi-task reinforcement learning and federated learning.

📄 PDF Abstract BibTeX arXiv:2006.13460

Code (0)

등록된 구현이 없습니다.

Tasks

Federated Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation

2026-05-19 · Ilya Levin, Maksim Shuklin, Eric Moulines, Paul Mangold 외 arxiv

In this paper, we establish Berry-Esseen-type bounds for federated linear stochastic approximation (LSA). Our results provide the first federated Gaussian approximations for LSA that explicitly capture communication-comp…

Federated Majorize-Minimization: Beyond Parameter Aggregation

2025-07-23 · Aymeric Dieuleveut, Gersende Fort, Mahmoud Hegazy, Hoi-To Wai arxiv

This paper proposes a unified approach for designing stochastic optimization algorithms that robustly scale to the federated learning setting. Our work studies a class of Majorize-Minimization (MM) problems, which posses…

Stochastic Optimization Federated Learning

Federated Stochastic Approximation under Markov Noise and Heterogeneity: Applications in Reinforcement Learning

2022-06-21 · Sajad Khodadadian, Pranay Sharma, Gauri Joshi, Siva Theja Maguluri

Since reinforcement learning algorithms are notoriously data-intensive, the task of sampling observations from the environment is usually split across multiple agents. However, transferring these observations from the ag…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation

2023-06-02 · Davoud Ataee Tarzanagh, Mingchen Li, Pranay Sharma, Samet Oymak

Stochastic approximation with multiple coupled sequences (MSA) has found broad applications in machine learning as it encompasses a rich class of problems including bilevel optimization (BLO), multi-level compositional o…

Bilevel Optimization

Federated Stochastic Gradient Langevin Dynamics

2020-04-23 · Khaoula El Mekkaoui, Diego Mesquita, Paul Blomstedt, Samuel Kaski

Stochastic gradient MCMC methods, such as stochastic gradient Langevin dynamics (SGLD), employ fast but noisy gradient estimates to enable large-scale posterior sampling. Although we can easily extend SGLD to distributed…

Federated Learning Metric Learning