Papers Distributed Optimization
“Distributed Optimization” 태그가 달린 논문 601편 · 필터 해제
Distributed Optimization for Over-Parameterized Learning
Distributed optimization often consists of two updating phases: local optimization and inter-node communication. Conventional approaches require working nodes to communicate with the server every one or few iterations to…
Deep LearningDistributed OptimizationThe Communication Complexity of Optimization
We consider the communication complexity of a number of distributed optimization problems. We start with the problem of solving a linear system. Suppose there is a coordinator together with $s$ servers $P_1, \ldots, P_s$…
Distributed OptimizationCommunication-Efficient Accurate Statistical Estimation
When the data are stored in a distributed manner, direct application of traditional statistical inference procedures is often prohibitive due to communication cost and privacy concerns. This paper develops and investigat…
Distributed OptimizationQsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations
Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including differen…
Distributed OptimizationQuantizationPowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compression schemes either do not scale well …
Distributed OptimizationDeep Learning for Distributed Optimization: Applications to Wireless Resource Management
This paper studies a deep learning (DL) framework to solve distributed non-convex constrained optimizations in wireless networks where multiple computing nodes, interconnected via backhaul links, desire to determine an e…
BinarizationDeep LearningDistributed OptimizationManagement+1Accelerated Sparsified SGD with Error Feedback
A stochastic gradient method for synchronous distributed optimization is studied. For reducing communication cost, we particularly focus on utilization of compression of communicated gradients. Several work has shown tha…
Distributed OptimizationDistributed estimation of the inverse Hessian by determinantal averaging
In distributed optimization and distributed numerical linear algebra, we often encounter an inversion bias: if we want to compute a quantity that depends on the inverse of a sum of distributed matrices, then the sum of t…
Distributed OptimizationUncertainty QuantificationLeader Stochastic Gradient Descent for Distributed Training of Deep Learning Models: Extension
We consider distributed optimization under communication constraints for training deep learning models. We propose a new algorithm, whose parameter updates rely on two forces: a regular gradient step, and a corrective di…
Distributed OptimizationOverSketched Newton: Fast Convex Optimization for Serverless Systems
Motivated by recent developments in serverless systems for large-scale computation as well as improvements in scalable randomized matrix algorithms, we develop OverSketched Newton, a randomized Hessian-based optimization…
Distributed OptimizationByzantine Fault Tolerant Distributed Linear Regression
This paper considers the problem of Byzantine fault tolerance in distributed linear regression in a multi-agent system. However, the proposed algorithms are given for a more general class of distributed optimization prob…
Distributed OptimizationregressionDifferentially Private Consensus-Based Distributed Optimization
Data privacy is an important concern in learning, when datasets contain sensitive information about individuals. This paper considers consensus-based distributed optimization under data privacy constraints. Consensus-bas…
Distributed OptimizationSLSGD: Secure and Efficient Distributed On-device Machine Learning
We consider distributed on-device learning with limited communication and security requirements. We propose a new robust distributed optimization algorithm with efficient communication and attack tolerance. The proposed …
BIG-bench Machine LearningData PoisoningDistributed OptimizationA Provably Communication-Efficient Asynchronous Distributed Inference Method for Convex and Nonconvex Problems
This paper proposes and analyzes a communication-efficient distributed optimization framework for general nonconvex nonsmooth signal processing and machine learning problems under an asynchronous protocol. At each iterat…
Distributed OptimizationOn Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communication
In distributed optimization and machine learning, multiple nodes coordinate to solve large problems. To do this, the nodes need to compress important algorithm information to bits so that it can be communicated over a di…
BIG-bench Machine LearningDistributed OptimizationSparsity Constrained Distributed Unmixing of Hyperspectral Data
Spectral unmixing (SU) is a technique to characterize mixed pixels in hyperspectral images measured by remote sensors. Most of the spectral unmixing algorithms are developed using the linear mixing models. To estimate en…
Distributed OptimizationQuantized Frank-Wolfe: Faster Optimization, Lower Communication, and Projection Free
How can we efficiently mitigate the overhead of gradient communications in distributed optimization? This problem is at the heart of training scalable machine learning models and has been mainly studied in the unconstrai…
Distributed OptimizationQuantizationStochastic OptimizationDo Subsampled Newton Methods Work for High-Dimensional Data?
Subsampled Newton methods approximate Hessian matrices through subsampling techniques, alleviating the cost of forming Hessian matrices but using sufficient curvature information. However, previous results require $\Omeg…
Distributed OptimizationVocal Bursts Intensity PredictionPredict Globally, Correct Locally: Parallel-in-Time Optimal Control of Neural Networks
The links between optimal control of dynamical systems and neural networks have proved beneficial both from a theoretical and from a practical point of view. Several researchers have exploited these links to investigate …
Distributed OptimizationRobust Learning from Untrusted Sources
Modern machine learning methods often require more data for training than a single expert can provide. Therefore, it has become a standard procedure to collect data from external sources, e.g. via crowdsourcing. Unfortun…
Distributed OptimizationLearning Theory