Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking
This paper considers the problem of decentralized optimization on compact submanifolds, where a finite sum of smooth (possibly non-convex) local functions is minimized by $n$ agents forming an undirected and connected graph. However, the efficiency of distributed optimization is often hindered by communication bottlenecks. To mitigate this, we propose the Quantized Riemannian Gradient Tracking (Q-RGT) algorithm, where agents update their local variables using quantized gradients. The introduction of quantization noise allows our algorithm to bypass the constraints of the accurate Riemannian projection operator (such as retraction), further improving iterative efficiency. To the best of our knowledge, this is the first algorithm to achieve an $\mathcal{O}(1/K)$ convergence rate in the presence of quantization, matching the convergence rate of methods without quantization. Additionally, we explicitly derive lower bounds on decentralized consensus associated with a function of quantization levels. Numerical experiments demonstrate that Q-RGT performs comparably to non-quantized methods while reducing communication bottlenecks and computational overhead.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed OptimizationQuantizationSimilar Papers 제목 키워드 기반
A Riemannian smoothing steepest descent method for non-Lipschitz optimization on submanifolds
In this paper, we propose a Riemannian smoothing steepest descent method to minimize a nonconvex and non-Lipschitz function on submanifolds. The generalized subdifferentials on Riemannian manifold and the Riemannian grad…
Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data
Many machine learning tasks, such as principal component analysis and low-rank matrix completion, give rise to manifold optimization problems. Although there is a large body of work studying the design and analysis of al…
Computational EfficiencyFederated LearningLow-Rank Matrix CompletionMatrix CompletionWeakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods
We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are ubiquitous in engineering applications …
Dictionary LearningVocal Bursts Type PredictionRiemannian-geometry-based modeling and clustering of network-wide non-stationary time series: The brain-network case
This paper advocates Riemannian multi-manifold modeling in the context of network-wide non-stationary time-series analysis. Time-series data, collected sequentially over time and across a network, yield features which ar…
ClusteringTime SeriesTime Series AnalysisManifold learning in Wasserstein space
This paper aims at building the theoretical foundations for manifold learning algorithms in the space of absolutely continuous probability measures $\mathcal{P}_{\mathrm{a.c.}}(\Omega)$ with $\Omega$ a compact and convex…