FedSplit: An algorithmic framework for fast federated optimization

2020-05-11 · NeurIPS 2020 12 · Reese Pathak, Martin J. Wainwright

Motivated by federated learning, we consider the hub-and-spoke model of distributed optimization in which a central authority coordinates the computation of a solution among many agents while limiting communication. We first study some past procedures for federated optimization, and show that their fixed points need not correspond to stationary points of the original optimization problem, even in simple convex settings with deterministic updates. In order to remedy these issues, we introduce FedSplit, a class of algorithms based on operator splitting procedures for solving distributed convex minimization with additive structure. We prove that these procedures have the correct fixed points, corresponding to optima of the original optimization problem, and we characterize their convergence rates under different settings. Our theory shows that these methods are provably robust to inexact computation of intermediate local quantities. We complement our theory with some simple experiments that demonstrate the benefits of our methods in practice.

📄 PDF Abstract BibTeX arXiv:2005.05238

Code (0)

등록된 구현이 없습니다.

Tasks

Distributed Optimization Federated Learning

Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data

2023-12-07 · Feifei Wang, Huiyun Tang, Yang Li

Federated learning is an emerging distributed machine learning framework aiming at protecting data privacy. Data heterogeneity is one of the core challenges in federated learning, which could severely degrade the converg…

Federated Learning Personalized Federated Learning

FedSplitX: Federated Split Learning for Computationally-Constrained Heterogeneous Clients

2023-10-23 · Jiyun Shin, JinHyun Ahn, Honggu Kang, Joonhyuk Kang

Foundation models (FMs) have demonstrated remarkable performance in machine learning but demand extensive training data and computational resources. Federated learning (FL) addresses the challenges posed by FMs, especial…

Federated Learning

FedSPLIT: One-Shot Federated Recommendation System Based on Non-negative Joint Matrix Factorization and Knowledge Distillation

2022-05-04 · Maksim E. Eren, Luke E. Richards, Manish Bhattarai, Roberto Yus 외

Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on the…

Collaborative Filtering Federated Learning Knowledge Distillation Privacy Preserving

Federated Split BERT for Heterogeneous Text Classification

2022-05-26 · Zhengyang Li, Shijing Si, Jianzong Wang, Jing Xiao

Pre-trained BERT models have achieved impressive performance in many natural language processing (NLP) tasks. However, in many real-world situations, textual data are usually decentralized over many clients and unable to…

Classification Federated Learning Quantization text-classification+1

Fast Convergence Algorithm for Analog Federated Learning

2020-10-30 · Shuhao Xia, Jingyang Zhu, Yuhan Yang, Yong Zhou 외

In this paper, we consider federated learning (FL) over a noisy fading multiple access channel (MAC), where an edge server aggregates the local models transmitted by multiple end devices through over-the-air computation …