A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions
In this paper the stability theorem of Borkar and Meyn is extended to include the case when the mean field is a differential inclusion. Two different sets of sufficient conditions are presented that guarantee the stability and convergence of stochastic recursive inclusions. Our work builds on the works of Benaim, Hofbauer and Sorin as well as Borkar and Meyn. As a corollary to one of the main theorems, a natural generalization of the Borkar and Meyn Theorem follows. In addition, the original theorem of Borkar and Meyn is shown to hold under slightly relaxed assumptions. Finally, as an application to one of the main theorems we discuss a solution to the approximate drift problem.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Convergence of Stochastic Approximation via Martingale and Converse Lyapunov Methods
In this paper, we study the almost sure boundedness and the convergence of the stochastic approximation (SA) algorithm. At present, most available convergence proofs are based on the ODE method, and the almost sure bound…
Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays
We generalize the Borkar-Meyn stability Theorem (BMT) to distributed stochastic approximations (SAs) with information delays that possess an arbitrary moment bound. To model the delays, we introduce Age of Information Pr…
The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise
Stochastic approximation is a class of algorithms that update a vector iteratively, incrementally, and stochastically, including, e.g., stochastic gradient descent and temporal difference learning. One fundamental challe…
reinforcement-learningReinforcement LearningA Note on Stability in Asynchronous Stochastic Approximation without Communication Delays
In this paper, we study asynchronous stochastic approximation algorithms without communication delays. Our main contribution is a stability proof for these algorithms that extends a method of Borkar and Meyn by accommoda…
reinforcement-learningReinforcement LearningAsynchronous Stochastic Approximation and Average-Reward Reinforcement Learning
This paper studies asynchronous stochastic approximation (SA) algorithms and their theoretical application to reinforcement learning in semi-Markov decision processes (SMDPs) with an average-reward criterion. We first ex…
Q-Learningreinforcement-learningReinforcement Learning