Self-Triggered Markov Decision Processes

2021-02-17 · Yunhan Huang, Quanyan Zhu

In this paper, we study Markov Decision Processes (MDPs) with self-triggered strategies, where the idea of self-triggered control is extended to more generic MDP models. This extension broadens the application of self-triggering policies to a broader range of systems. We study the co-design problems of the control policy and the triggering policy to optimize two pre-specified cost criteria. The first cost criterion is introduced by incorporating a pre-specified update penalty into the traditional MDP cost criteria to reduce the use of communication resources. Under this criteria, a novel dynamic programming (DP) equation called DP equation with optimized lookahead to proposed to solve for the self-triggering policy under this criteria. The second self-triggering policy is to maximize the triggering time while still guaranteeing a pre-specified level of sub-optimality. Theoretical underpinnings are established for the computation and implementation of both policies. Through a gridworld numerical example, we illustrate the two policies' effectiveness in reducing sources consumption and demonstrate the trade-offs between resource consumption and system performance.

📄 PDF Abstract BibTeX arXiv:2102.08571

Code (0)

등록된 구현이 없습니다.

Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach

2020-01-28 · Ashkan Haji Hosseinloo, Alexander Ryzhov, Aldo Bischi, Henni Ouerdane 외

Smart buildings have great potential for shaping an energy-efficient, sustainable, and more economic future for our planet as buildings account for approximately 40% of the global energy consumption. Future of the smart …

Decision Making Reinforcement Learning Reinforcement Learning (RL)

Factorized Machine Self-Confidence for Decision-Making Agents

2018-10-15 · Brett W. Israelsen, Nisar R. Ahmed, Eric Frew, Dale Lawrence 외

Algorithmic assurances from advanced autonomous systems assist human users in understanding, trusting, and using such systems appropriately. Designing these systems with the capacity of assessing their own capabilities i…

Decision Making Decision Making Under Uncertainty Reinforcement Learning

A Lifting Approach to Learning-Based Self-Triggered Control with Gaussian Processes

2022-02-21 · Wang Zhijun, Kazumune Hashimoto, Wataru Hashimoto, Shigemasa Takai

This paper investigates the design of self-triggered control for networked control systems (NCS), where the dynamics of the plant is unknown apriori. To deal with the nature of the self-triggered control, in which state …

Gaussian Processes

Reinforcement Learning with Action-Triggered Observations

2025-10-02 · Alexander Ryabchenko, Wenlong Mou arxiv

We introduce Action-Triggered Sporadically Traceable Markov Decision Processes (ATST-MDPs), a reinforcement learning framework for partial observability in which full state observations occur stochastically at each step,…

Reinforcement Learning

A Notation for Markov Decision Processes

2015-12-30 · Philip S. Thomas, Billy Okal

This paper specifies a notation for Markov decision processes.