Game-theoretical control with continuous action sets

2014-12-01 · Steven Perkins, Panayotis Mertikopoulos, David S. Leslie

Motivated by the recent applications of game-theoretical learning techniques to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets, and we propose an actor-critic reinforcement learning algorithm that provably converges to equilibrium in this class of problems. The method employed is to analyse the learning process under study through a mean-field dynamical system that evolves in an infinite-dimensional function space (the space of probability distributions over the players' continuous controls). To do so, we extend the theory of finite-dimensional two-timescale stochastic approximation to an infinite-dimensional, Banach space setting, and we prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provably-convergent learning algorithm in which players do not need to keep track of the controls selected by the other agents.

📄 PDF Abstract BibTeX arXiv:1412.0543

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

A0C: Alpha Zero in Continuous Action Space

2018-05-24 · Thomas M. Moerland, Joost Broekens, Aske Plaat, Catholijn M. Jonker

A core novelty of Alpha Zero is the interleaving of tree search and deep learning, which has proven very successful in board games like Chess, Shogi and Go. These games have a discrete action space. However, many real-wo…

Board Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Learning to Design Games: Strategic Environments in Reinforcement Learning

2017-07-05 · Haifeng Zhang, Jun Wang, Zhiming Zhou, Wei-Nan Zhang 외

In typical reinforcement learning (RL), the environment is assumed given and the goal of the learning is to identify an optimal policy for the agent taking actions through its interactions with the environment. In this p…

Game Design reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning in Cournot Games

2020-09-14 · Yuanyuan Shi, Baosen Zhang

In this work, we study the interaction of strategic agents in continuous action Cournot games with limited information feedback. Cournot game is the essential market model for many socio-economic systems where agents lea…

continuous-control Continuous Control Multi-agent Reinforcement Learning reinforcement-learning+2

Factored Action Spaces in Deep Reinforcement Learning

2021-01-01 · Thomas Pierrot, Valentin Macé, Jean-Baptiste Sevestre, Louis Monier 외

Very large action spaces constitute a critical challenge for deep Reinforcement Learning (RL) algorithms. An existing approach consists in splitting the action space into smaller components and choosing either independen…

continuous-control Continuous Control Deep Reinforcement Learning Dota 2+4

Flow Matching for Offline Reinforcement Learning with Discrete Actions

2026-02-05 · Fairoz Nower Khan, Nabuat Zaman Nahim, Ruiquan Huang, Haibo Yang 외 arxiv

Generative policies based on diffusion models and flow matching have shown strong promise for offline reinforcement learning (RL), but their applicability remains largely confined to continuous action spaces. To address …

Reinforcement Learning Offline RL