Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning
Recently, local SGD has got much attention and been extensively studied in the distributed learning community to overcome the communication bottleneck problem. However, the superiority of local SGD to minibatch SGD only …
Distributed OptimizationFederated LearningConcentration of Non-Isotropic Random Tensors with Applications to Learning and Empirical Risk Minimization
Dimension is an inherent bottleneck to some modern learning tasks, where optimization methods suffer from the size of the data. In this paper, we study non-isotropic distributions of data and develop tools that aim at re…
Distributed OptimizationDelayed Projection Techniques for Linearly Constrained Problems: Convergence Rates, Acceleration, and Applications
In this work, we study a novel class of projection-based algorithms for linearly constrained problems (LCPs) which have a lot of applications in statistics, optimization, and machine learning. Conventional primal gradien…
Distributed OptimizationPrivacy PreservingCost-efficient SVRG with Arbitrary Sampling
We consider the problem of distributed optimization over a network, using a stochastic variance reduced gradient (SVRG) algorithm, where executing every iteration of the algorithm requires computation and exchange of gra…
Distributed OptimizationConvergent Adaptive Gradient Methods in Decentralized Optimization
Adaptive gradient methods including Adam, AdaGrad, and their variants have been very successful for training deep learning models, such as neural networks, in the past few years. Meanwhile, given the need for distributed…
Distributed OptimizationDesign of heterogeneous multi-agent system for distributed computation
A group behavior of a heterogeneous multi-agent system is studied which obeys an "average of individual vector fields" under strong couplings among the agents. Under stability of the averaged dynamics (not asking stabili…
Distributed OptimizationState EstimationFairness-Oriented User Scheduling for Bursty Downlink Transmission Using Multi-Agent Reinforcement Learning
In this work, we develop practical user scheduling algorithms for downlink bursty traffic with emphasis on user fairness. In contrast to the conventional scheduling algorithms that either equally divides the transmission…
Distributed OptimizationFairnessMulti-agent Reinforcement LearningReinforcement Learning (RL)+1Byzantine-Resilient Non-Convex Stochastic Gradient Descent
We study adversary-resilient stochastic distributed optimization, in which $m$ machines can independently compute stochastic gradients, and cooperate to jointly optimize over their local objective functions. However, an …
Distributed OptimizationLinear Convergence of Distributed Mirror Descent with Integral Feedback for Strongly Convex Problems
Distributed optimization often requires finding the minimum of a global objective function written as a sum of local functions. A group of agents work collectively to minimize the global function. We study a continuous-t…
Distributed OptimizationWyner-Ziv Estimators for Distributed Mean Estimation with Side Information and Optimization
Communication efficient distributed mean estimation is an important primitive that arises in many distributed learning and optimization scenarios such as federated learning. Without any probabilistic assumptions on the u…
Distributed OptimizationFederated LearningSparse sketches with small inversion bias
For a tall $n\times d$ matrix $A$ and a random $m\times n$ sketching matrix $S$, the sketched estimate of the inverse covariance matrix $(A^\top A)^{-1}$ is typically biased: $E[(\tilde A^\top\tilde A)^{-1}]\ne(A^\top A)…
Distributed OptimizationDistributed Stochastic Consensus Optimization with Momentum for Nonconvex Nonsmooth Problems
While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approa…
Distributed OptimizationWidely-distributed Radar Imaging Based on Consensus ADMM
A widely-distributed radar system is a promising architecture to enhance radar imaging performance. However, most existing algorithms rely on isotropic scattering assumption, which is only satisfied in collocated radar s…
Distributed OptimizationDiversityDistributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
This paper focuses on the distributed optimization of stochastic saddle point problems. The first part of the paper is devoted to lower bounds for the centralized and decentralized distributed methods for smooth (strongl…
Distributed OptimizationFederated LearningDistributed Resource Allocation with Multi-Agent Deep Reinforcement Learning for 5G-V2V Communication
We consider the distributed resource selection problem in Vehicle-to-vehicle (V2V) communication in the absence of a base station. Each vehicle autonomously selects transmission resources from a pool of shared resources …
Deep Reinforcement LearningDistributed OptimizationMulti-agent Reinforcement Learningreinforcement-learning+2Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. Several works have analyzed the convergence of f…
Distributed OptimizationFederated LearningSelection biasByzantine-Robust Learning on Heterogeneous Datasets via Resampling
In Byzantine-robust distributed optimization, a central server wants to train a machine learning model over data distributed across multiple workers. However, a fraction of these workers may deviate from the prescribed a…
Distributed OptimizationAsynchronous Distributed Optimization with Stochastic Delays
We study asynchronous finite sum minimization in a distributed-data setting with a central parameter server. While asynchrony is well understood in parallel settings where the data is accessible by all machines -- e.g., …
Distributed OptimizationDistributed Mirror Descent with Integral Feedback: Asymptotic Convergence Analysis of Continuous-time Dynamics
This work addresses distributed optimization, where a network of agents wants to minimize a global strongly convex objective function. The global function can be written as a sum of local convex functions, each of which …
Distributed OptimizationGraph neural networks-based Scheduler for Production planning problems using Reinforcement Learning
Reinforcement learning (RL) is increasingly adopted in job shop scheduling problems (JSSP). But RL for JSSP is usually done using a vectorized representation of machine features as the state space. It has three major pro…
Distributed OptimizationGraph Neural NetworkJob Shop Schedulingreinforcement-learning+2