Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization
Communication overhead is one of the key challenges that hinders the scalability of distributed optimization algorithms. In this paper, we study local distributed SGD, where data is partitioned among computation nodes, a…
Distributed OptimizationGPUAsynchronous Decentralized SGD with Quantized and Local Updates
Decentralized optimization is emerging as a viable alternative for scalable distributed machine learning, but also introduces new challenges in terms of synchronization costs. To this end, several communication-reduction…
BlockingDistributed OptimizationImage ClassificationQuantizationAccelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks
This paper proposes a novel family of primal-dual-based distributed algorithms for smooth, convex, multi-agent optimization over networks that uses only gradient information and gossip communications. The algorithms can …
Distributed OptimizationSparsification as a Remedy for Staleness in Distributed Asynchronous SGD
Large scale machine learning is increasingly relying on distributed optimization, whereby several machines contribute to the training process of a statistical model. In this work we study the performance of asynchronous,…
Distributed OptimizationSCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Federated Averaging (FedAvg) has emerged as the algorithm of choice for federated learning due to its simplicity and low communication cost. However, in spite of recent research efforts, its performance is not fully unde…
Distributed OptimizationFederated LearningSlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
Distributed optimization is essential for training large models on large datasets. Multiple approaches have been proposed to reduce the communication overhead in distributed training, such as synchronizing only after per…
BlockingDistributed Optimizationimage-classificationImage Classification+2PopSGD: Decentralized Stochastic Gradient Descent in the Population Model
The population model is a standard way to represent large-scale decentralized distributed systems, in which agents with limited computational power interact in randomly chosen pairs, in order to collectively solve global…
Distributed OptimizationSchedulingGradient-Consensus: Linearly Convergent Distributed Optimization Algorithm over Directed Graphs
In this article, we propose a new approach, optimize then agree for minimizing a sum $ f = \sum_{i=1}^n f_i(x)$ of convex objective functions over a directed graph. The optimize then agree approach decouples the optimiza…
Distributed OptimizationCommunication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction
There is growing interest in large-scale machine learning and optimization over decentralized networks, e.g. in the context of multi-agent learning and federated learning. Due to the imminent need to alleviate the commun…
Distributed OptimizationFederated LearningConvex Set Disjointness, Distributed Learning of Halfspaces, and LP Feasibility
We study the Convex Set Disjointness (CSD) problem, where two players have input sets taken from an arbitrary fixed domain~$U\subseteq \mathbb{R}^d$ of size $\lvert U\rvert = n$. Their mutual goal is to decide using mini…
Distributed OptimizationLEMMAProximal gradient flow and Douglas-Rachford splitting dynamics: global exponential stability via integral quadratic constraints
Many large-scale and distributed optimization problems can be brought into a composite form in which the objective function is given by the sum of a smooth term and a nonsmooth regularizer. Such problems can be solved vi…
Distributed OptimizationFederated Learning: Challenges, Methods, and Future Directions
Federated learning involves training statistical models over remote devices or siloed data centers, such as mobile phones or hospitals, while keeping data localized. Training in heterogeneous and potentially massive netw…
BIG-bench Machine LearningDistributed OptimizationFederated LearningPrivacy PreservingGradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimization
Optimization is at the heart of machine learning, statistics and many applied scientific disciplines. It also has a long history in physics, ranging from the minimal action principle to finding ground states of disordere…
BIG-bench Machine LearningDistributed OptimizationStochastic OptimizationPopt4jlib: A Parallel/Distributed Optimization Library for Java
This paper describes the architectural design as well as key implementation details of the Open Source popt4jlib library (https://githhub.org/ioannischristou/popt4jlib) that contains a fairly large number of meta-heurist…
Distributed OptimizationCentralised and Distributed Optimization for Aggregated Flexibility Services Provision
The recent deployment of distributed battery units in prosumer premises offer new opportunities for providing aggregated flexibility services to both distribution system operators and balance responsible parties. The opt…
Distributed OptimizationData Encoding for Byzantine-Resilient Distributed Optimization
We study distributed optimization in the presence of Byzantine adversaries, where both data and computation are distributed among $m$ worker machines, $t$ of which may be corrupt. The compromised nodes may collaborativel…
Distributed OptimizationTrading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization
Communication overhead is one of the key challenges that hinder the scalability of distributed optimization algorithms to train large neural networks. In recent years, there has been a great deal of research to alleviate…
Distributed OptimizationDiversityAsymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning
We provide a discussion of several recent results which, in certain scenarios, are able to overcome a barrier in distributed stochastic optimization for machine learning. Our focus is the so-called asymptotic network ind…
BIG-bench Machine LearningDistributed OptimizationStochastic OptimizationDistributed Optimization for Smart Cyber-Physical Networks
The presence of embedded electronics and communication capabilities as well as sensing and control in smart devices has given rise to the novel concept of cyber-physical networks, in which agents aim at cooperatively sol…
Distributed Optimizationglobal-optimizationSecure Architectures Implementing Trusted Coalitions for Blockchained Distributed Learning (TCLearn)
Distributed learning across a coalition of organizations allows the members of the coalition to train and share a model without sharing the data used to optimize this model. In this paper, we propose new secure architect…
Distributed Optimization