Distributed TD(0) with Almost No Communication
We provide a new non-asymptotic analysis of distributed temporal difference learning with linear function approximation. Our approach relies on ``one-shot averaging,'' where $N$ agents run identical local copies of the TD(0) method and average the outcomes only once at the very end. We demonstrate a version of the linear time speedup phenomenon, where the convergence time of the distributed process is a factor of $N$ faster than the convergence time of TD(0). This is the first result proving benefits from parallelism for temporal difference methods.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Distributed Inference with Sparse and Quantized Communication
We consider the problem of distributed inference where agents in a network observe a stream of private signals generated by an unknown state, and aim to uniquely identify this state from a finite set of hypotheses. We fo…
QuantizationSignal-Comparison-Based Distributed Estimation Under Decaying Average Data Rate Communications
The paper investigates the distributed estimation problem under low bit rate communications. Based on the signal-comparison (SC) consensus protocol under binary-valued communications, a new consensus+innovations type dis…
Event-Triggered Distributed Estimation With Decaying Communication Rate
We study distributed estimation of a high-dimensional static parameter vector through a group of sensors whose communication network is modeled by a fixed directed graph. Different from existing time-triggered communicat…
Communication-Efficient Projection-Free Algorithm for Distributed Optimization
Distributed optimization has gained a surge of interest in recent years. In this paper we propose a distributed projection free algorithm named Distributed Conditional Gradient Sliding(DCGS). Compared to the state-of-the…
Distributed OptimizationMatrix CompletionQuantized Epoch-SGD for Communication-Efficient Distributed Learning
Due to its efficiency and ease to implement, stochastic gradient descent (SGD) has been widely used in machine learning. In particular, SGD is one of the most popular optimization methods for distributed learning. Recent…
Quantization