Actor-Critic Algorithms

1999-12-01 · Advances in Neural Information Processing Systems 1999 12 · Vijay Konda, John Tsitsiklis

We propose and analyze a class of actor-critic algorithms for simulation-based optimization of a Markov decision process over a parameterized family of randomized stationary policies. These are two-time-scale algorithms in which the critic uses TD learning with a linear approximation architecture and the actor is updated in an approximate gradient direction based on information provided by the critic. We show that the features for the critic should span a subspace prescribed by the choice of parameterization of the actor. We conclude by discussing convergence properties and some open problems.

📄 PDF Abstract BibTeX

Code (1)

xiuyu0000/new_papers_codes/tree/main/ac mindspore

Stackelberg Actor-Critic: Game-Theoretic Reinforcement Learning Algorithms

2021-09-25 · Liyuan Zheng, Tanner Fiez, Zane Alumbaugh, Benjamin Chasnov 외

The hierarchical interaction between the actor and critic in actor-critic based reinforcement learning algorithms naturally lends itself to a game-theoretic interpretation. We adopt this viewpoint and model the actor and…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Projected Natural Actor-Critic

2013-12-01 · NeurIPS 2013 12 · Philip S. Thomas, William C. Dabney, Stephen Giguere, Sridhar Mahadevan

Natural actor-critics are a popular class of policy search algorithms for finding locally optimal policies for Markov decision processes. In this paper we address a drawback of natural actor-critics that limits their rea…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Finite-Time Analysis of Three-Timescale Constrained Actor-Critic and Constrained Natural Actor-Critic Algorithms

2023-10-25 · Prashansa Panda, Shalabh Bhatnagar

Actor Critic methods have found immense applications on a wide range of Reinforcement Learning tasks especially when the state-action space is large. In this paper, we consider actor critic and natural actor critic algor…

Double Actor-Critic with TD Error-Driven Regularization in Reinforcement Learning

2024-09-28 · Haohui Chen, Zhiyong Chen, Aoxiang Liu, Wentuo Fang

To obtain better value estimation in reinforcement learning, we propose a novel algorithm based on the double actor-critic framework with temporal difference error-driven regularization, abbreviated as TDDR. TDDR employs…

continuous-control Continuous Control

Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL

2021-02-23 · Siddharth Mysore, Bassel Mabsout, Renato Mancuso, Kate Saenko

Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures. This case study explores the performance impact of network sizes when con…

Reinforcement Learning (RL)