Policy Synthesis and Reinforcement Learning for Discounted LTL
The difficulty of manually specifying reward functions has led to an interest in using linear temporal logic (LTL) to express objectives for reinforcement learning (RL). However, LTL has the downside that it is sensitive to small perturbations in the transition probabilities, which prevents probably approximately correct (PAC) learning without additional assumptions. Time discounting provides a way of removing this sensitivity, while retaining the high expressivity of the logic. We study the use of discounted LTL for policy synthesis in Markov decision processes with unknown transition probabilities, and show how to reduce discounted LTL to discounted-sum reward via a reward machine when all discount factors are identical.
Code (0)
등록된 구현이 없습니다.
Tasks
PAC learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)SensitivitySimilar Papers 제목 키워드 기반
Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning
We present a reinforcement learning (RL) framework to synthesize a control policy from a given linear temporal logic (LTL) specification in an unknown stochastic environment that can be modeled as a Markov Decision Proce…
Motion Planningreinforcement-learningReinforcement LearningReinforcement Learning (RL)On the Convergence of Discounted Policy Gradient Methods
Many popular policy gradient methods for reinforcement learning follow a biased approximation of the policy gradient known as the discounted approximation. While it has been shown that the discounted approximation of the…
Policy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Stability-Constrained Markov Decision Processes Using MPC
In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We wil…
Model Predictive ControlLearning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewards
As the operations of autonomous systems generally affect simultaneously several users, it is crucial that their designs account for fairness considerations. In contrast to standard (deep) reinforcement learning (RL), we …
Deep Reinforcement LearningFairnessreinforcement-learningReinforcement Learning (RL)Analyzing and Bridging the Gap between Maximizing Total Reward and Discounted Reward in Deep Reinforcement Learning
In deep reinforcement learning applications, maximizing discounted reward is often employed instead of maximizing total reward to ensure the convergence and stability of algorithms, even though the performance metric for…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning