paper-with-me

Papers

Asynchronous Adaptation and Learning over Networks - Part II: Performance Analysis

2013-12-19 · Xiaochuan Zhao, Ali H. Sayed

In Part I \cite{Zhao13TSPasync1}, we introduced a fairly general model for asynchronous events over adaptive networks including random topologies, random link failures, random data arrival times, and agents turning on and off randomly. We performed a stability analysis and established the notable fact that the network is still able to converge in the mean-square-error sense to the desired solution. Once stable behavior is guaranteed, it becomes important to evaluate how fast the iterates converge and how close they get to the optimal solution. This is a demanding task due to the various asynchronous events and due to the fact that agents influence each other. In this Part II, we carry out a detailed analysis of the mean-square-error performance of asynchronous strategies for solving distributed optimization and adaptation problems over networks. We derive analytical expressions for the mean-square convergence rate and the steady-state mean-square-deviation. The expressions reveal how the various parameters of the asynchronous behavior influence network performance. In the process, we establish the interesting conclusion that even under the influence of asynchronous events, all agents in the adaptive network can still reach an $O(\nu^{1 + \gamma_o'})$ near-agreement with some $\gamma_o' > 0$ while approaching the desired solution within $O(\nu)$ accuracy, where $\nu$ is proportional to the small step-size parameter for adaptation.

📄 PDF Abstract BibTeX arXiv:1312.5438

Code (0)

등록된 구현이 없습니다.

Tasks

Distributed Optimization

Similar Papers 제목 키워드 기반

Asynchronous Adaptation and Learning over Networks - Part III: Comparison Analysis

2013-12-19 · Xiaochuan Zhao, Ali H. Sayed

In Part II [3] we carried out a detailed mean-square-error analysis of the performance of asynchronous adaptation and learning over networks under a fairly general model for asynchronous events including random topologie…

Asynchronous Adaptation and Learning over Networks --- Part I: Modeling and Stability Analysis

2013-12-19 · Xiaochuan Zhao, Ali H. Sayed

In this work and the supporting Parts II [2] and III [3], we provide a rather detailed analysis of the stability and performance of asynchronous strategies for solving distributed optimization and adaptation problems ove…

Distributed Optimization

$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning

2023-06-14 · NeurIPS 2023 11 · Adel Nabli, Eugene Belilovsky, Edouard Oyallon

Distributed training of Deep Learning models has been critical to many recent successes in the field. Current standard methods primarily rely on synchronous centralized algorithms which induce major communication bottlen…

Asynchronous adaptive networks

2015-11-30 · Ali H. Sayed, Xiaochuan Zhao

In a recent article [1] we surveyed advances related to adaptation, learning, and optimization over synchronous networks. Various distributed strategies were discussed that enable a collection of networked agents to inte…

Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization

2019-09-25 · Vyacheslav Kungurtsev, Malcolm Egan, Bapi Chatterjee, Dan Alistarh

Asynchronous distributed methods are a popular way to reduce the communication and synchronization costs of large-scale optimization. Yet, for all their success, little is known about their convergence guarantees in the …

Scheduling