Decentralized Feature-Distributed Optimization for Generalized Linear Models
We consider the "all-for-one" decentralized learning problem for generalized linear models. The features of each sample are partitioned among several collaborating agents in a connected network, but only one agent observes the response variables. To solve the regularized empirical risk minimization in this distributed setting, we apply the Chambolle--Pock primal--dual algorithm to an equivalent saddle-point formulation of the problem. The primal and dual iterations are either in closed-form or reduce to coordinate-wise minimization of scalar convex functions. We establish convergence rates for the empirical risk minimization under two different assumptions on the loss function (Lipschitz and square root Lipschitz), and show how they depend on the characteristics of the design matrix and the Laplacian of the network.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed OptimizationSimilar Papers 제목 키워드 기반
Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
Decentralized optimization has become a fundamental tool for large-scale learning systems; however, most existing methods rely on the classical Lipschitz smoothness assumption, which is often violated in problems with ra…
Distributed OptimizationA Semi-Decentralized Tikhonov-based Algorithm for Optimal Generalized Nash Equilibrium Selection
To optimally select a generalized Nash equilibrium, in this paper, we propose a semi-decentralized algorithm based on a double-layer Tikhonov regularization method. Technically, we extend the Tikhonov method for equilibr…
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM
In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censo…
QuantizationA Multi-Agent Primal-Dual Strategy for Composite Optimization over Distributed Features
This work studies multi-agent sharing optimization problems with the objective function being the sum of smooth local functions plus a convex (possibly non-smooth) function coupling all agents. This scenario arises in ma…
regressionCommunication-Efficient Federated Non-Linear Bandit Optimization
Federated optimization studies the problem of collaborative function optimization among multiple clients (e.g. mobile devices or organizations) under the coordination of a central server. Since the data is collected sepa…