Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Distributed Optimization, Averaging via ADMM, and Network Topology
There has been an increasing necessity for scalable optimization methods, especially due to the explosion in the size of datasets and model complexity in modern machine learning applications. Scalable solvers often distr…
Distributed OptimizationOn Communication Compression for Distributed Optimization on Heterogeneous Data
Lossy gradient compression, with either unbiased or biased compressors, has become a key tool to avoid the communication bottleneck in centrally coordinated distributed training of machine learning models. We analyze the…
Distributed OptimizationGTAdam: Gradient Tracking with Adaptive Momentum for Distributed Online Optimization
This paper deals with a network of computing agents aiming to solve an online optimization problem in a distributed fashion, i.e., by means of local computation and communication, without any central coordinator. We prop…
Distributed OptimizationGeneral Classificationimage-classificationImage Classification+1Projected Push-Sum Gradient Descent-Ascent for Convex Optimizationwith Application to Economic Dispatch Problems
We propose a novel algorithm for solving convex, constrained and distributed optimization problems defined on multi-agent-networks, where each agent has exclusive access to a part of the global objective function. The ag…
Distributed OptimizationDistributed Personalized Gradient Tracking with Convex Parametric Models
We present a distributed optimization algorithm for solving online personalized optimization problems over a network of computing and communicating nodes, each of which linked to a specific user. The local objective func…
Distributed OptimizationGraph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis
Deep learning has recently emerged as a disruptive technology to solve challenging radio resource management problems in wireless networks. However, the neural network architectures adopted by existing works suffer from …
Computational EfficiencyDistributed OptimizationGPUManagementDistributed Training of Graph Convolutional Networks
The aim of this work is to develop a fully-distributed algorithmic framework for training graph convolutional networks (GCNs). The proposed method is able to exploit the meaningful relational structure of the input data,…
Distributed OptimizationA Unified Linear Speedup Analysis of Federated Averaging and Nesterov FedAvg
Federated learning (FL) learns a model jointly from a set of participating devices without sharing each other's privately held data. The characteristics of non-i.i.d. data across the network, low device participation, hi…
Distributed OptimizationFederated LearningDistributed optimization for nonrigid nano-tomography
Resolution level and reconstruction quality in nano-computed tomography (nano-CT) are in part limited by the stability of microscopes, because the magnitude of mechanical vibrations during scanning becomes comparable to …
Distributed OptimizationOptical Flow EstimationFederated Learning with Compression: Unified Analysis and Sharp Guarantees
In federated learning, communication cost is often a critical bottleneck to scale up distributed optimization algorithms to collaboratively learn a model from millions of devices with potentially unreliable or limited co…
Distributed OptimizationFederated LearningLinear Convergent Decentralized Optimization with Compression
Communication compression has become a key strategy to speed up distributed optimization. However, existing decentralized algorithms with compression mainly focus on compressing DGD-type algorithms. They are unsatisfacto…
Distributed OptimizationDistributed Model Predictive Control with Reconfigurable Terminal Ingredients for Reference Tracking
Various efforts have been devoted to developing stabilizing distributed Model Predictive Control (MPC) schemes for tracking piecewise constant references. In these schemes, terminal sets are usually computed offline and …
Distributed OptimizationModel Predictive ControlAdvances in Asynchronous Parallel and Distributed Optimization
Motivated by large-scale optimization problems arising in the context of machine learning, there have been several advances in the study of asynchronous parallel and distributed optimization methods during the past decad…
Distributed OptimizationStochastic OptimizationDistributed Collision-Free Motion Coordination on a Sphere: A Conic Control Barrier Function Approach
This letter studies a distributed collision avoidance control problem for a group of rigid bodies on a sphere. A rigid body network, consisting of multiple rigid bodies constrained to a spherical surface and an interconn…
Collision AvoidanceDistributed OptimizationDEED: A General Quantization Scheme for Communication Efficiency in Bits
In distributed optimization, a popular technique to reduce communication is quantization. In this paper, we provide a general analysis framework for inexact gradient descent that is applicable to quantization schemes. We…
Distributed OptimizationFederated LearningQuantizationApproximate Gradient Coding with Optimal Decoding
In distributed optimization problems, a technique called gradient coding, which involves replicating data points, has been used to mitigate the effect of straggling machines. Recent work has studied approximate gradient …
Distributed OptimizationByzantine-Robust Learning on Heterogeneous Datasets via Bucketing
In Byzantine robust distributed or federated learning, a central server wants to train a machine learning model over data distributed across multiple workers. However, a fraction of these workers may deviate from the pre…
Distributed OptimizationFederated LearningFederated Accelerated Stochastic Gradient Descent
We propose Federated Accelerated Stochastic Gradient Descent (FedAc), a principled acceleration of Federated Averaging (FedAvg, also known as Local SGD) for distributed optimization. FedAc is the first provable accelerat…
Distributed OptimizationDistributed Newton Can Communicate Less and Resist Byzantine Workers
We develop a distributed second order optimization algorithm that is communication-efficient as well as robust against Byzantine failures of the worker machines. We propose COMRADE (COMunication-efficient and Robust Appr…
Distributed OptimizationAnytime MiniBatch: Exploiting Stragglers in Online Distributed Optimization
Distributed optimization is vital in solving large-scale machine learning problems. A widely-shared feature of distributed optimization techniques is the requirement that all nodes complete their assigned tasks in each c…
Distributed Optimization