Dynamic Knowledge Injection for AIXI Agents

2023-12-18 · Samuel Yang-Zhao, Kee Siong Ng, Marcus Hutter

Prior approximations of AIXI, a Bayesian optimality notion for general reinforcement learning, can only approximate AIXI's Bayesian environment model using an a-priori defined set of models. This is a fundamental source of epistemic uncertainty for the agent in settings where the existence of systematic bias in the predefined model class cannot be resolved by simply collecting more data from the environment. We address this issue in the context of Human-AI teaming by considering a setup where additional knowledge for the agent in the form of new candidate models arrives from a human operator in an online fashion. We introduce a new agent called DynamicHedgeAIXI that maintains an exact Bayesian mixture over dynamically changing sets of models via a time-adaptive prior constructed from a variant of the Hedge algorithm. The DynamicHedgeAIXI agent is the richest direct approximation of AIXI known to date and comes with good performance guarantees. Experimental results on epidemic control on contact networks validates the agent's practical utility.

📄 PDF Abstract BibTeX arXiv:2312.16184

Code (0)

등록된 구현이 없습니다.

Tasks

General Reinforcement Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

AIXIjs: A Software Demo for General Reinforcement Learning

2017-05-22 · John Aslanides

Reinforcement learning is a general and powerful framework with which to study and implement artificial intelligence. Recent advances in deep learning have enabled RL algorithms to achieve impressive performance in restr…

General Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+2

Quantum AIXI: Universal Intelligence via Quantum Information

2025-05-27 · Elija Perrier

AIXI is a widely studied model of artificial general intelligence (AGI) based upon principles of induction and reinforcement learning. However, AIXI is fundamentally classical in nature - as are the environments in which…

Reinforcement Learning via AIXI Approximation

2010-07-13 · AAAI 2010 2010 7 · Joel Veness, Kee Siong Ng, Marcus Hutter, David Silver

This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. This approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcem…

General Reinforcement Learning Open-Ended Question Answering reinforcement-learning Reinforcement Learning+1

Universal AI maximizes Variational Empowerment

2025-02-20 · Yusuke Hayashi, Koichi Takahashi

This paper presents a theoretical framework unifying AIXI -- a model of universal AI -- with variational empowerment as an intrinsic drive for exploration. We build on the existing framework of Self-AIXI -- a universal l…

A Monte Carlo AIXI Approximation

2009-09-04 · Joel Veness, Kee Siong Ng, Marcus Hutter, William Uther 외

This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. Our approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforceme…