Multi-Agent Reinforcement Learning with Temporal Logic Specifications

2021-02-01 · Lewis Hammond, Alessandro Abate, Julian Gutierrez, Michael Wooldridge

In this paper, we study the problem of learning to satisfy temporal logic specifications with a group of agents in an unknown environment, which may exhibit probabilistic behaviour. From a learning perspective these specifications provide a rich formal language with which to capture tasks or objectives, while from a logic and automated verification perspective the introduction of learning capabilities allows for practical applications in large, stochastic, unknown environments. The existing work in this area is, however, limited. Of the frameworks that consider full linear temporal logic or have correctness guarantees, all methods thus far consider only the case of a single temporal logic specification and a single agent. In order to overcome this limitation, we develop the first multi-agent reinforcement learning technique for temporal logic specifications, which is also novel in its ability to handle multiple specifications. We provide correctness and convergence guarantees for our main algorithm - ALMANAC (Automaton/Logic Multi-Agent Natural Actor-Critic) - even when using function approximation. Alongside our theoretical results, we further demonstrate the applicability of our technique via a set of preliminary experiments.

📄 PDF Abstract BibTeX arXiv:2102.00582

Code (1)

lrhammond/almanac 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications

2023-06-11 · Jiangwei Wang, Shuo Yang, Ziyan An, Songyang Han 외

Reward design is a key component of deep reinforcement learning, yet some tasks and designer's objectives may be unnatural to define as a scalar cost function. Among the various techniques, formal methods integrated with…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning

2022-05-25 · Geraud Nangue Tasse, Devon Jarvis, Steven James, Benjamin Rosman

It is desirable for an agent to be able to solve a rich variety of problems that can be specified through language in the same environment. A popular approach towards obtaining such agents is to reuse skills learned in p…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Instructing Goal-Conditioned Reinforcement Learning Agents with Temporal Logic Objectives

2023-09-21 · NeurIPS 2023 11

Goal-conditioned reinforcement learning (RL) is a powerful approach for learning general-purpose skills by reaching diverse goals. However, it has limitations when it comes to task-conditioned policies, where goals are s…

Using Logical Specifications of Objectives in Multi-Objective Reinforcement Learning

2019-10-03 · Kolby Nottingham, Anand Balakrishnan, Jyotirmoy Deshmukh, David Wingate

It is notoriously difficult to control the behavior of reinforcement learning agents. Agents often learn to exploit the environment or reward signal and need to be retrained multiple times. The multi-objective reinforcem…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents

2024-02-06 · Yash Shukla, Tanushree Burman, Abhishek Kulkarni, Robert Wright 외

Reinforcement Learning (RL) has made significant strides in enabling artificial agents to learn diverse behaviors. However, learning an effective policy often requires a large number of environment interactions. To mitig…

continuous-control Continuous Control Decision Making Q-Learning+4