Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization

2022-02-14 · Mirco Mutti, Riccardo De Santi, Emanuele Rossi, Juan Felipe Calderon, Michael Bronstein, Marcello Restelli

In the sequential decision making setting, an agent aims to achieve systematic generalization over a large, possibly infinite, set of environments. Such environments are modeled as discrete Markov decision processes with both states and actions represented through a feature vector. The underlying structure of the environments allows the transition dynamics to be factored into two components: one that is environment-specific and another that is shared. Consider a set of environments that share the laws of motion as an example. In this setting, the agent can take a finite amount of reward-free interactions from a subset of these environments. The agent then must be able to approximately solve any planning task defined over any environment in the original set, relying on the above interactions only. Can we design a provably efficient algorithm that achieves this ambitious goal of systematic generalization? In this paper, we give a partially positive answer to this question. First, we provide a tractable formulation of systematic generalization by employing a causal viewpoint. Then, under specific structural assumptions, we provide a simple learning algorithm that guarantees any desired planning error up to an unavoidable sub-optimality term, while showcasing a polynomial sample complexity.

📄 PDF Abstract BibTeX arXiv:2202.06545

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision MakingSystematic Generalization

Causality Meets Locality: Provably Generalizable and Scalable Policy Learning for Networked Systems

2025-10-24 · Hao Liang, Shuqing Shi, Yudi Zhang, Biwei Huang 외 arxiv

Large-scale networked systems, such as traffic, power, and wireless grids, challenge reinforcement-learning agents with both scale and environment shifts. To address these challenges, we propose GSAC (Generalizable and S…

Representation Learning Domain Generalization

Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications

2025-12-19 · Cristiano da Costa Cunha, Wei Liu, Tim French, Ajmal Mian arxiv

Integrating causal inference (CI) with reinforcement learning (RL) has emerged as a powerful paradigm to address critical limitations in classical RL, including low explainability, lack of robustness and generalization f…

Representation Learning Reinforcement Learning Transfer Learning Causal Inference

Certified Causal Attribution for Real-Time Attack Forensics in 6G Network Slicing

2026-05-26 · Minh K. Quan, Pubudu N. Pathirana arxiv

Cross-slice attack attribution in 6G networks requires identifying causal propagation chains through shared infrastructure in under 100 ms. Existing methods struggle to satisfy this strict SLA without sacrificing accurac…

Causal Reinforcement Learning: A Survey

2023-07-04 · Zhihong Deng, Jing Jiang, Guodong Long, Chengqi Zhang

Reinforcement learning is an essential paradigm for solving sequential decision problems under uncertainty. Despite many remarkable achievements in recent decades, applying reinforcement learning methods in the real worl…

reinforcement-learning Reinforcement Learning Survey Transfer Learning

Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search

2024-07-12 · Kim Hammar, Neil Dhir, Rolf Stadler

The CAGE-2 challenge is considered a standard benchmark to compare methods for autonomous cyber defense. Current state-of-the-art methods evaluated against this benchmark are based on model-free (offline) reinforcement l…