Communication Efficient Parallel Algorithms for Optimization on Manifolds
The last decade has witnessed an explosion in the development of models, theory and computational algorithms for "big data" analysis. In particular, distributed computing has served as a natural and dominating paradigm for statistical inference. However, the existing literature on parallel inference almost exclusively focuses on Euclidean data and parameters. While this assumption is valid for many applications, it is increasingly more common to encounter problems where the data or the parameters lie on a non-Euclidean space, like a manifold for example. Our work aims to fill a critical gap in the literature by generalizing parallel inference algorithms to optimization on manifolds. We show that our proposed algorithm is both communication efficient and carries theoretical convergence guarantees. In addition, we demonstrate the performance of our algorithm to the estimation of Fr\'echet means on simulated spherical data and the low-rank matrix completion problem over Grassmann manifolds applied to the Netflix prize data set.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed ComputingLow-Rank Matrix CompletionMatrix CompletionvalidSimilar Papers 제목 키워드 기반
An efficient Exact-PGA algorithm for constant curvature manifolds
Manifold-valued datasets are widely encountered in many computer vision tasks. A non-linear analog of the PCA, called the Principal Geodesic Analysis (PGA) suited for data lying on Riemannian manifolds was reported in li…
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RASA} achieves optimal sample complexities …
Stochastic OptimizationQuantum Manifold Optimization: A Design Framework for Future Communications Systems
Inspired by recent developments in various areas of science relevant to quantum computing, we introduce quantum manifold optimization (QMO) as a promising framework for solving constrained optimization problems in next-g…
On Parallel or Distributed Asynchronous Iterations with Unbounded Delays and Possible Out of Order Messages or Flexible Communication for Convex Optimization Problems and Machine Learning
We describe several features of parallel or distributed asynchronous iterative algorithms such as unbounded delays, possible out of order messages or flexible communication. We concentrate on the concept of macroiteratio…
SurveyCommunication-Efficient, 2D Parallel Stochastic Gradient Descent for Distributed-Memory Optimization
Distributed-memory implementations of numerical optimization algorithm, such as stochastic gradient descent (SGD), require interprocessor communication at every iteration of the algorithm. On modern distributed-memory cl…
Binary Classification