Neural Recursive Belief States in Multi-Agent Reinforcement Learning

2021-02-03 · Pol Moreno, Edward Hughes, Kevin R. McKee, Bernardo Avila Pires, Théophane Weber

In multi-agent reinforcement learning, the problem of learning to act is particularly difficult because the policies of co-players may be heavily conditioned on information only observed by them. On the other hand, humans readily form beliefs about the knowledge possessed by their peers and leverage beliefs to inform decision-making. Such abilities underlie individual success in a wide range of Markov games, from bluffing in Poker to conditional cooperation in the Prisoner's Dilemma, to convention-building in Bridge. Classical methods are usually not applicable to complex domains due to the intractable nature of hierarchical beliefs (i.e. beliefs of other agents' beliefs). We propose a scalable method to approximate these belief structures using recursive deep generative models, and to use the belief models to obtain representations useful to acting in complex tasks. Our agents trained with belief models outperform model-free baselines with equivalent representational capacity using common training paradigms. We also show that higher-order belief models outperform agents with lower-order models.

📄 PDF Abstract BibTeX arXiv:2102.02274

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning

2026-08-06 · Zi-Han Wang, Zhengxi Lu, Zhiyuan Yao, Jinyang Wu 외 hf

Reinforcement learning (RL) with verifiable rewards constructs trajectory-level advantage estimates, yet it often fails to credit the few pivotal decisions that determine outcomes in long-horizon, multi-turn agentic task…

Reinforcement Learning

Directly Forecasting Belief for Reinforcement Learning with Delays

2025-05-01 · Qingyuan Wu, Yuhui Wang, Simon Sinong Zhan, YiXuan Wang 외

Reinforcement learning (RL) with delays is challenging as sensory perceptions lag behind the actual events: the RL agent needs to estimate the real state of its environment based on past observations. State-of-the-art (S…

D4RL MuJoCo reinforcement-learning Reinforcement Learning+1

Sophisticated Inference

2020-06-07 · Karl Friston, Lancelot Da Costa, Danijar Hafner, Casper Hesp 외

Active inference offers a first principle account of sentient behaviour, from which special and important cases can be derived, e.g., reinforcement learning, active learning, Bayes optimal inference, Bayes optimal design…

Active Learning counterfactual Form

ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction

2025-12-23 · Aly Lidayan, Jakob Bjorner, Satvik Golechha, Kartik Goyal 외 arxiv

As the time horizons of sequential decision-making tasks grow, keeping full interaction histories in model context becomes increasingly costly. Recent work reduces context lengths by instead conditioning decision-making …

Belief States for Cooperative Multi-Agent Reinforcement Learning under Partial Observability

2025-04-11 · Paul J. Pritz, Kin K. Leung

Reinforcement learning in partially observable environments is typically challenging, as it requires agents to learn an estimate of the underlying system state. These challenges are exacerbated in multi-agent settings, w…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning