Agent policies from higher-order causal functions

2025-12-11 · Matt Wilson arxiv

We establish a correspondence between equivalence classes of agent-state policies for deterministic POMDPs and one-input process functions (the classical-deterministic limit of higher-order quantum operations). We use this correspondence to build a bridge between the agent-environment interaction in artificial intelligence, causal structure in the foundations of physics, and logic in computer science. We construct a *-autonomous category PF of types which supports an interpretation of one-step evaluation of policies, and multi-agent observation constraints, into cuts and monoidal products. In terms of types, we develop the correspondence further by identifying observation-independent decentralised POMDPs as the natural domain for the multi-input process functions used to model indefinite causality. We then prove a strict separation between general multi-input process function and definite-ordered process function performance on such dec-POMDPs, by finding an instance for which policies utilizing an indefinite causal structure can achieve greater finite-horizon rewards than policies which are restricted to a fixed background causal structure.

📄 PDF Abstract BibTeX arXiv:2512.10937

Code (0)

등록된 구현이 없습니다.

Causal Influence in Federated Edge Inference

2024-05-02 · Mert Kayaalp, Yunus Inan, Visa Koivunen, Ali H. Sayed

In this paper, we consider a setting where heterogeneous agents with connectivity are performing inference using unlabeled streaming data. Observed data are only partially informative about the target variable of interes…

Crowd Counting Decision Making

Potential-Based Advice for Stochastic Policy Learning

2019-07-20 · Baicen Xiao, Bhaskar Ramasubramanian, Andrew Clark, Hannaneh Hajishirzi 외

This paper augments the reward received by a reinforcement learning agent with potential functions in order to help the agent learn (possibly stochastic) optimal policies. We show that a potential-based reward shaping sc…

Q-Learning Reinforcement Learning

Tiered Reward: Designing Rewards for Specification and Fast Learning of Desired Behavior

2022-12-07 · Zhiyuan Zhou, Shreyas Sundara Raman, Henry Sowerby, Michael L. Littman

Reinforcement-learning agents seek to maximize a reward signal through environmental interactions. As humans, our job in the learning process is to design reward functions to express desired behavior and enable the agent…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Learning Multi-Level Hierarchies with Hindsight

2017-12-04 · Andrew Levy, George Konidaris, Robert Platt, Kate Saenko

Hierarchical agents have the potential to solve sequential decision making tasks with greater sample efficiency than their non-hierarchical counterparts because hierarchical agents can break down tasks into sets of subta…

Decision Making Hierarchical Reinforcement Learning Reinforcement Learning Sequential Decision Making

Q-Cogni: An Integrated Causal Reinforcement Learning Framework

2023-02-26 · Cris Cunha, Wei Liu, Tim French, Ajmal Mian

We present Q-Cogni, an algorithmically integrated causal reinforcement learning framework that redesigns Q-Learning with an autonomous causal structure discovery method to improve the learning process with causal inferen…

Causal Inference Decision Making Q-Learning reinforcement-learning+2