Specifying Non-Markovian Rewards in MDPs Using LDL on Finite Traces (Preliminary Version)

2017-06-25 · Ronen Brafman, Giuseppe De Giacomo, Fabio Patrizi

In Markov Decision Processes (MDPs), the reward obtained in a state depends on the properties of the last state and action. This state dependency makes it difficult to reward more interesting long-term behaviors, such as always closing a door after it has been opened, or providing coffee only following a request. Extending MDPs to handle such non-Markovian reward function was the subject of two previous lines of work, both using variants of LTL to specify the reward function and then compiling the new model back into a Markovian model. Building upon recent progress in the theories of temporal logics over finite traces, we adopt LDLf for specifying non-Markovian rewards and provide an elegant automata construction for building a Markovian model, which extends that of previous work and offers strong minimality and compositionality guarantees.

📄 PDF Abstract BibTeX arXiv:1706.08100

Code (0)

등록된 구현이 없습니다.

Using Experience Classification for Training Non-Markovian Tasks

2023-10-18 · Ruixuan Miao, Xu Lu, Cong Tian, Bin Yu 외

Unlike the standard Reinforcement Learning (RL) model, many real-world tasks are non-Markovian, whose rewards are predicated on state history rather than solely on the current state. Solving a non-Markovian task, frequen…

Autonomous Driving Classification Medical Diagnosis Reinforcement Learning (RL)

Expressive Temporal Specifications for Reward Monitoring

2025-11-16 · Omar Adalat, Francesco Belardinelli arxiv

Specifying informative and dense reward functions remains a pivotal challenge in Reinforcement Learning, as it directly affects the efficiency of agent training. In this work, we harness the expressive power of quantitat…

Reinforcement Learning Decision Making

Learning Non-Markovian Reward Models in MDPs

2020-01-25 · Gavin Rens, Jean-François Raskin

There are situations in which an agent should receive rewards only after having accomplished a series of previous tasks. In other words, the reward that the agent receives is non-Markovian. One natural and quite general …

Active Learning

Learning and Solving Regular Decision Processes

2020-03-02 · Eden Abadi, Ronen I. Brafman

Regular Decision Processes (RDPs) are a recently introduced model that extends MDPs with non-Markovian dynamics and rewards. The non-Markovian behavior is restricted to depend on regular properties of the history. These …

Clustering

Reinforcement Learning with Non-Markovian Rewards

2019-12-05 · Maor Gaon, Ronen I. Brafman

The standard RL world model is that of a Markov Decision Process (MDP). A basic premise of MDPs is that the rewards depend on the last state and action only. Yet, many real-world rewards are non-Markovian. For example, a…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)