Criticality-Based Varying Step-Number Algorithm for Reinforcement Learning

2022-01-13 · Yitzhak Spielberg, Amos Azaria

In the context of reinforcement learning we introduce the concept of criticality of a state, which indicates the extent to which the choice of action in that particular state influences the expected return. That is, a state in which the choice of action is more likely to influence the final outcome is considered as more critical than a state in which it is less likely to influence the final outcome. We formulate a criticality-based varying step number algorithm (CVS) - a flexible step number algorithm that utilizes the criticality function provided by a human, or learned directly from the environment. We test it in three different domains including the Atari Pong environment, Road-Tree environment, and Shooter environment. We demonstrate that CVS is able to outperform popular learning algorithms such as Deep Q-Learning and Monte Carlo.

📄 PDF Abstract BibTeX arXiv:2201.05034

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

The Concept of Criticality in Reinforcement Learning

2018-10-16 · Yitzhak Spielberg, Amos Azaria

Reinforcement learning methods carry a well known bias-variance trade-off in n-step algorithms for optimal control. Unfortunately, this has rarely been addressed in current research. This trade-off principle holds indepe…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

CARL: Criticality-Aware Agentic Reinforcement Learning

2025-12-04 · Leyang Shen, Yang Zhang, Chun Kai Ling, Xiaoyan Zhao 외 arxiv

Agents capable of accomplishing complex tasks through multiple interactions with the environment have emerged as a popular research direction. However, in such multi-step settings, the conventional group-level policy opt…

Reinforcement Learning

Improving Mixed-Criticality Scheduling with Reinforcement Learning

2025-04-04 · Muhammad El-Mahdy, Nourhan Sakr, Rodrigo Carrasco

This paper introduces a novel reinforcement learning (RL) approach to scheduling mixed-criticality (MC) systems on processors with varying speeds. Building upon the foundation laid by [1], we extend their work to address…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Scheduling

Enhancing Adaptive Mixed-Criticality Scheduling with Deep Reinforcement Learning

2024-11-01 · Bruno Mendes, Pedro F. Souto, Pedro C. Diniz

Adaptive Mixed-Criticality (AMC) is a fixed-priority preemptive scheduling algorithm for mixed-criticality hard real-time systems. It dominates many other scheduling algorithms for mixed-criticality systems, but does so …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Scheduling

Criticality and Safety Margins for Reinforcement Learning

2024-09-26 · Alexander Grushin, Walt Woods, Alvaro Velasquez, Simon Khan

State of the art reinforcement learning methods sometimes encounter unsafe situations. Identifying when these situations occur is of interest both for post-hoc analysis and during deployment, where it might be advantageo…

reinforcement-learning Reinforcement Learning