Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Accelerating Distributed Optimization: A Primal-Dual Perspective on Local Steps
In distributed machine learning, efficient training across multiple agents with different data distributions poses significant challenges. Even with a centralized coordinator, current algorithms that achieve optimal comm…
Distributed OptimizationGraph Neural Networks Gone Hogwild
Message passing graph neural networks (GNNs) would appear to be powerful tools to learn distributed algorithms via gradient descent, but generate catastrophically incorrect predictions when nodes update asynchronously du…
Distributed OptimizationDistributed Utility Optimization in Vehicular Communication Systems
In this paper, we study the problem of utility maximization in the uplink of vehicle-to-infrastructure communication systems. The studied scenarios consider four practical aspects of mobile radio communication links: i) …
Distributed OptimizationA KL-based Analysis Framework with Applications to Non-Descent Optimization Methods
We propose a novel analysis framework for non-descent-type optimization methodologies in nonconvex scenarios based on the Kurdyka-Lojasiewicz property. Our framework allows covering a broad class of algorithms, including…
Distributed OptimizationACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
Training LLMs relies on distributed implementations using multiple GPUs to compute gradients in parallel with sharded optimizers. However, synchronizing gradients in data parallel setups introduces communication overhead…
Distributed OptimizationFederated LearningGPULocal Methods with Adaptivity via Scaling
The rapid development of machine learning and deep learning has introduced increasingly complex optimization challenges that must be addressed. Indeed, training modern, advanced models has become difficult to implement w…
Distributed OptimizationFederated LearningLog-Scale Quantization in Distributed First-Order Methods: Gradient-based Learning from Distributed Data
Decentralized strategies are of interest for learning from large-scale data over networks. This paper studies learning over a network of geographically distributed nodes/agents subject to quantization. Each node possesse…
Distributed OptimizationQuantizationDifferentially-Private Distributed Model Predictive Control of Linear Discrete-Time Systems with Global Constraints
Distributed model predictive control (DMPC) has attracted extensive attention as it can explicitly handle system constraints and achieve optimal control in a decentralized manner. However, the deployment of DMPC strategi…
Distributed OptimizationModel Predictive ControlMicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
We propose a new variant of the Adam optimizer called MicroAdam that specifically minimizes memory overheads, while maintaining theoretical convergence guarantees. We achieve this by compressing the gradient information …
Distributed OptimizationThe Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
Local SGD is a popular optimization method in distributed learning, often outperforming other algorithms in practice, including mini-batch SGD. Despite this success, theoretically proving the dominance of local SGD in se…
Distributed OptimizationFlattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
We propose a novel algorithm for distributed stochastic gradient descent (SGD) with compressed gradient communication in the parameter-server framework. Our gradient compression technique, named flattened one-bit stochas…
Distributed OptimizationQuantizationStructured Reinforcement Learning for Incentivized Stochastic Covert Optimization
This paper studies how a stochastic gradient algorithm (SG) can be controlled to hide the estimate of the local stationary point from an eavesdropper. Such problems are of significant interest in distributed optimization…
Distributed OptimizationFederated Learningreinforcement-learningReinforcement LearningDistributed Traffic Signal Control via Coordinated Maximum Pressure-plus-Penalty
This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy feat…
Computational EfficiencyDistributed OptimizationTraffic Signal ControlEstimation Network Design framework for efficient distributed optimization
Distributed decision problems features a group of agents that can only communicate over a peer-to-peer network, without a central memory. In applications such as network control and data ranking, each agent is only affec…
Distributed OptimizationRate Analysis of Coupled Distributed Stochastic Approximation for Misspecified Optimization
We consider an $n$ agents distributed optimization problem with imperfect information characterized in a parametric sense, where the unknown parameter can be solved by a distinct distributed parameter learning problem. T…
Distributed OptimizationDistributed Fractional Bayesian Learning for Adaptive Optimization
This paper considers a distributed adaptive optimization problem, where all agents only have access to their local cost functions with a common unknown parameter, whereas they mean to collaboratively estimate the true pa…
Distributed OptimizationFederated Optimization with Doubly Regularized Drift Correction
Federated learning is a distributed optimization paradigm that allows training machine learning models across decentralized devices while keeping the data localized. The standard method, FedAvg, suffers from client drift…
Distributed OptimizationFederated LearningPIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory System
Modern Machine Learning (ML) training on large-scale datasets is a very time-consuming workload. It relies on the optimization algorithm Stochastic Gradient Descent (SGD) due to its effectiveness, simplicity, and general…
CPUDistributed OptimizationGPUGeneralized Gradient Descent is a Hypergraph Functor
Cartesian reverse derivative categories (CRDCs) provide an axiomatic generalization of the reverse derivative, which allows generalized analogues of classic optimization algorithms such as gradient descent to be applied …
Distributed OptimizationDistributed Maximum Consensus over Noisy Links
We introduce a distributed algorithm, termed noise-robust distributed maximum consensus (RD-MC), for estimating the maximum value within a multi-agent network in the presence of noisy communication links. Our approach en…
Distributed Optimization