Model-based Reinforcement Learning from Signal Temporal Logic Specifications

2020-11-10 · Parv Kapoor, Anand Balakrishnan, Jyotirmoy V. Deshmukh

Techniques based on Reinforcement Learning (RL) are increasingly being used to design control policies for robotic systems. RL fundamentally relies on state-based reward functions to encode desired behavior of the robot and bad reward functions are prone to exploitation by the learning agent, leading to behavior that is undesirable in the best case and critically dangerous in the worst. On the other hand, designing good reward functions for complex tasks is a challenging problem. In this paper, we propose expressing desired high-level robot behavior using a formal specification language known as Signal Temporal Logic (STL) as an alternative to reward/cost functions. We use STL specifications in conjunction with model-based learning to design model predictive controllers that try to optimize the satisfaction of the STL specification over a finite time horizon. The proposed algorithm is empirically evaluated on simulations of robotic system such as a pick-and-place robotic arm, and adaptive cruise control for autonomous vehicles.

📄 PDF Abstract BibTeX arXiv:2011.04950

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous VehiclesModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning from Demonstrations using Signal Temporal Logic

2021-02-15 · Aniruddh G. Puranic, Jyotirmoy V. Deshmukh, Stefanos Nikolaidis

Learning-from-demonstrations is an emerging paradigm to obtain effective robot control policies for complex tasks via reinforcement learning without the need to explicitly design reward functions. However, it is suscepti…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications

2023-06-11 · Jiangwei Wang, Shuo Yang, Ziyan An, Songyang Han 외

Reward design is a key component of deep reinforcement learning, yet some tasks and designer's objectives may be unnatural to define as a scalar cost function. Among the various techniques, formal methods integrated with…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Tractable Reinforcement Learning of Signal Temporal Logic Objectives

2020-01-26 · L4DC 2020 6 · Harish Venkataraman, Derya Aksaray, Peter Seiler

Signal temporal logic (STL) is an expressive language to specify time-bound real-world robotic tasks and safety specifications. Recently, there has been an interest in learning optimal policies to satisfy STL specificati…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learning

2022-11-30 · Naman Saxena, Gorantla Sandeep, Pushpak Jagtap

Signal Temporal Logic (STL) is a powerful framework for describing the complex temporal and logical behaviour of the dynamical system. Numerous studies have attempted to employ reinforcement learning to learn a controlle…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning with Temporal Logic Specifications

2021-02-01 · Lewis Hammond, Alessandro Abate, Julian Gutierrez, Michael Wooldridge

In this paper, we study the problem of learning to satisfy temporal logic specifications with a group of agents in an unknown environment, which may exhibit probabilistic behaviour. From a learning perspective these spec…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)