CARL: Criticality-Aware Agentic Reinforcement Learning

2025-12-04 · Leyang Shen, Yang Zhang, Chun Kai Ling, Xiaoyan Zhao, Tat-Seng Chua arxiv

Agents capable of accomplishing complex tasks through multiple interactions with the environment have emerged as a popular research direction. However, in such multi-step settings, the conventional group-level policy optimization algorithm becomes suboptimal because of its underlying assumption that each step holds equal contribution, which deviates significantly from reality. Our analysis reveals that only the action choices on a small fraction of states are critical in determining the final outcome. Building on this insight, we propose CARL, a criticality-aware reinforcement learning algorithm tailored for long-horizon agentic reasoning. CARL leverages entropy as a heuristic proxy for state criticality and achieves focused training by assigning rewards to actions taken from high-criticality states while excluding actions taken from low-criticality states from model updates, avoiding noisy credit assignment and redundant computation. Extensive experiments demonstrate that CARL achieves both stronger performance and higher efficiency across diverse evaluation settings. The source code will be publicly available.

📄 PDF Abstract BibTeX arXiv:2512.04949

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Criticality-Based Varying Step-Number Algorithm for Reinforcement Learning

2022-01-13 · Yitzhak Spielberg, Amos Azaria

In the context of reinforcement learning we introduce the concept of criticality of a state, which indicates the extent to which the choice of action in that particular state influences the expected return. That is, a st…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

From Prompts to Pavement: LMMs-based Agentic Behavior-Tree Generation Framework for Autonomous Vehicles

2026-01-18 · Omar Y. Goba, Ahmed Y. Gado, Catherine M. Elias, Ahmed Hussein arxiv

Autonomous vehicles (AVs) require adaptive behavior planners to navigate unpredictable, real-world environments safely. Traditional behavior trees (BTs) offer structured decision logic but are inherently static and deman…

Autonomous Vehicles

Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents

2026-06-03 · Dae Yon Hwang, Raunaq Suri, Valentin Villecroze, Anthony L. Caterini 외 arxiv

LLM agents operate in two distinct regimes: open-weight agents amenable to reinforcement learning (RL) and black-box agents whose behaviour must be controlled purely at test time. Although black-box agents are often back…

Reinforcement Learning Bayesian Inference

Shackling Uncertainty using Mixed Criticality in Monte-Carlo Tree Search

2024-07-17 · Franco Cordeiro, Samuel Tardieu, Laurent Pautet

In the world of embedded systems, optimizing actions with the uncertain costs of multiple resources is a complex challenge. Existing methods include plan building based on Monte Carlo Tree Search (MCTS), an approach that…

Enhancing Adaptive Mixed-Criticality Scheduling with Deep Reinforcement Learning

2024-11-01 · Bruno Mendes, Pedro F. Souto, Pedro C. Diniz

Adaptive Mixed-Criticality (AMC) is a fixed-priority preemptive scheduling algorithm for mixed-criticality hard real-time systems. It dominates many other scheduling algorithms for mixed-criticality systems, but does so …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Scheduling