Distributed Mini-Batch SDCA

2015-07-29 · Martin Takáč, Peter Richtárik, Nathan Srebro

We present an improved analysis of mini-batched stochastic dual coordinate ascent for regularized empirical loss minimization (i.e. SVM and SVM-type objectives). Our analysis allows for flexible sampling schemes, including where data is distribute across machines, and combines a dependence on the smoothness of the loss and/or the data spread (measured through the spectral norm).

📄 PDF Abstract BibTeX arXiv:1507.08322

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SVM A Support Vector Machine, or SVM, is a non-parametric supervised learning model. For non-linear classification and regression, they utilise the kernel trick to map inputs…

Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization

2015-10-22 · Xi He, Martin Takáč

In this paper we develop dual free mini-batch SDCA with adaptive probabilities for regularized empirical risk minimization. This work is motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method, however, …

Accelerated Mini-Batch Stochastic Dual Coordinate Ascent

2013-05-12 · NeurIPS 2013 12 · Shai Shalev-Shwartz, Tong Zhang

Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is of…

BIG-bench Machine Learning

Randomized Dual Coordinate Ascent with Arbitrary Sampling

2014-11-21 · Zheng Qu, Peter Richtárik, Tong Zhang

We study the problem of minimizing the average of a large number of smooth convex functions penalized with a strongly convex regularizer. We propose and analyze a novel primal-dual method (Quartz) which at every iteratio…

Communication-Efficient Distributed Dual Coordinate Ascent

2014-09-04 · NeurIPS 2014 12 · Martin Jaggi, Virginia Smith, Martin Takáč, Jonathan Terhorst 외

Communication remains the most significant bottleneck in the performance of distributed optimization algorithms for large-scale machine learning. In this paper, we propose a communication-efficient framework, CoCoA, that…

BIG-bench Machine Learning Distributed Optimization

Analysis of Distributed Stochastic Dual Coordinate Ascent

2013-12-04 · Tianbao Yang, Shenghuo Zhu, Rong Jin, Yuanqing Lin

In \citep{Yangnips13}, the author presented distributed stochastic dual coordinate ascent (DisDCA) algorithms for solving large-scale regularized loss minimization. Extraordinary performances have been observed and repor…