Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness
Language model training in distributed settings is limited by the communication cost of gradient exchanges. In this short note, we extend recent work from Malladi et al. (2023), using shared randomness to perform distrib…
Distributed OptimizationLanguage ModelingLanguage ModellingRobust Optimization, Structure/Control co-design, Distributed Optimization, Monolithic Optimization, Robust Control, Parametric Uncertainty
This paper presents an end-to-end framework for robust structure/control optimization of an industrial benchmark. When dealing with space structures, a reduction of the spacecraft mass is paramount to minimize the missio…
Distributed OptimizationTowards Scalable Multi-View Reconstruction of Geometry and Materials
In this paper, we propose a novel method for joint recovery of camera pose, object geometry and spatially-varying Bidirectional Reflectance Distribution Function (svBRDF) of 3D scenes that exceed object-scale and hence c…
Distributed OptimizationUnbiased Compression Saves Communication in Distributed Optimization: When and How Much?
Communication compression is a common technique in distributed optimization that can alleviate communication overhead by transmitting compressed gradients and model parameters. However, compression can introduce informat…
Distributed OptimizationError Feedback Shines when Features are Rare
We provide the first proof that gradient descent $\left({\color{green}\sf GD}\right)$ with greedy sparsification $\left({\color{green}\sf TopK}\right)$ and error feedback $\left({\color{green}\sf EF}\right)$ can obtain b…
Distributed OptimizationAn Equivalent Circuit Approach to Distributed Optimization
Distributed optimization is an essential paradigm to solve large-scale optimization problems in modern applications where big-data and high-dimensionality creates a computational bottleneck. Distributed optimization algo…
Distributed OptimizationNumerical IntegrationCoordinated Frequency-Constrained Stochastic Economic Dispatch for Integrated Transmission and Distribution System via Distributed Optimization
When large-scale uncertain centralized and distributed renewable energy sources are connected to a power system, separate dispatching of the transmission power system (TPS) and the active distribution network (ADN) will …
Distributed OptimizationQ-SHED: Distributed Optimization at the Edge via Hessian Eigenvectors Quantization
Edge networks call for communication efficient (low overhead) and robust distributed optimization (DO) algorithms. These are, in fact, desirable qualities for DO frameworks, such as federated edge learning techniques, in…
Distributed OptimizationQuantizationDualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regime
We propose a new training algorithm, named DualFL (Dualized Federated Learning), for solving distributed optimization problems in federated learning. DualFL achieves communication acceleration for very general convex cos…
Computational EfficiencyDistributed OptimizationFederated LearningOn the Finite-Time Behavior of Suboptimal Linear Model Predictive Control
Inexact methods for model predictive control (MPC), such as real-time iterative schemes or time-distributed optimization, alleviate the computational burden of exact MPC by providing suboptimal solutions. While the asymp…
Distributed OptimizationModel Predictive ControlFederated TD Learning over Finite-Rate Erasure Channels: Linear Speedup under Markovian Sampling
Federated learning (FL) has recently gained much attention due to its effectiveness in speeding up supervised learning tasks under communication and privacy constraints. However, whether similar speedups can be establish…
Distributed OptimizationFederated LearningQuantizationreinforcement-learning+1Network-GIANT: Fully distributed Newton-type optimization via harmonic Hessian consensus
This paper considers the problem of distributed multi-agent learning, where the global aim is to minimize a sum of local objective (empirical loss) functions through local optimization and information exchange between ne…
Distributed OptimizationFederated LearningVocal Bursts Type PredictionDistributed Coordination of Multi-Microgrids in Active Distribution Networks for Provisioning Ancillary Services
With the phenomenal growth in renewable energy generation, the conventional synchronous generator-based power plants are gradually getting replaced by renewable energy sources-based microgrids. Such transition gives rise…
Distributed OptimizationA Novel Decentralized Algorithm for Coordinating the Optimal Power and Traffic Flows with EVs based on Variable Inner Loop Selection
The electric power distribution network (PDN) and the transportation network (TN) are generally operated/coordinated by different entities. However, they are coupled with each other due to electric vehicle charging stati…
Distributed OptimizationStochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis
We study finite-sum distributed optimization problems involving a master node and $n-1$ local nodes under the popular $\delta$-similarity and $\mu$-strong convexity conditions. We propose two new algorithms, SVRS and Acc…
Distributed OptimizationD-SVM over Networked Systems with Non-Ideal Linking Conditions
This paper considers distributed optimization algorithms, with application in binary classification via distributed support-vector-machines (D-SVM) over multi-agent networks subject to some link nonlinearities. The agent…
Binary ClassificationDistributed OptimizationQuantizationCEC: Crowdsourcing-based Evolutionary Computation for Distributed Optimization
Crowdsourcing is an emerging computing paradigm that takes advantage of the intelligence of a crowd to solve complex problems effectively. Besides collecting and processing data, it is also a great demand for the crowd t…
Distributed OptimizationA Survey on Distributed Evolutionary Computation
The rapid development of parallel and distributed computing paradigms has brought about great revolution in computing. Thanks to the intrinsic parallelism of evolutionary computation (EC), it is natural to implement EC o…
Distributed ComputingDistributed OptimizationSurveyDistributed Optimization for Quadratic Cost Functions over Large-Scale Networks with Quantized Communication and Finite-Time Convergence
We propose two distributed iterative algorithms that can be used to solve, in finite time, the distributed optimization problem over quadratic local cost functions in large-scale networks. The first algorithm exhibits sy…
Distributed OptimizationFederated LearningQuantizationSchedulingOn Degeneracy Issues in Multi-parametric Programming and Critical Region Exploration based Distributed Optimization in Smart Grid Operations
Improving renewable energy resource utilization efficiency is crucial to reducing carbon emissions, and multi-parametric programming has provided a systematic perspective in conducting analysis and optimization toward th…
Computational EfficiencyDistributed OptimizationScheduling