홈 › Papers

Communication-Efficient Distributed Dual Coordinate Ascent

2014-09-04 · NeurIPS 2014 12 · Martin Jaggi, Virginia Smith, Martin Takáč, Jonathan Terhorst, Sanjay Krishnan, Thomas Hofmann, Michael. I. Jordan

Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that uses local computation in a primal-dual setting to dramatically reduce the amount of necessary communication. We provide a strong convergence rate analysis for this class of algorithms, as well as experiments on real-world distributed datasets with implementations in Spark. In our experiments, we find that as compared to state-of-the-art mini-batch versions of SGD and SDCA algorithms, CoCoA converges to the same .001-accurate solution quality on average 25x as quickly.

📄 PDF Abstract BibTeX arXiv:1409.1458

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Distributed Optimization

Methods 이 논문이 사용한 방법론

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

Distributed Dual Coordinate Ascent in General Tree Networks and Communication Network Effect on Synchronous Machine Learning

2017-03-14 · Myung Cho, Lifeng Lai, Weiyu Xu

Due to the big size of data and limited data storage volume of a single computer or a single server, data are often stored in a distributed manner. Thus, performing large-scale machine learning operations with the distri…

BIG-bench Machine Learning

Trading Computation for Communication: Distributed Stochastic Dual Coordinate Ascent

2013-12-01 · NeurIPS 2013 12 · Tianbao Yang

We present and study a distributed optimization algorithm by employing a stochastic dual coordinate ascent method. Stochastic dual coordinate ascent methods enjoy strong theoretical guarantees and often have better perf…

Distributed Optimization

Distributed Dual Coordinate Ascent with Imbalanced Data on a General Tree Network

2023-08-28 · Myung Cho, Lifeng Lai, Weiyu Xu

In this paper, we investigate the impact of imbalanced data on the convergence of distributed dual coordinate ascent in a tree network for solving an empirical loss minimization problem in distributed machine learning. T…

Distributed Asynchronous Dual Free Stochastic Dual Coordinate Ascent

2016-05-29 · Zhouyuan Huo, Heng Huang

The primal-dual distributed optimization methods have broad large-scale machine learning applications. Previous primal-dual distributed methods are not applicable when the dual formulation is not available, e.g. the sum-…