A Strongly Asymptotically Optimal Agent in General Environments

2019-03-04 · Michael K. Cohen, Elliot Catt, Marcus Hutter

Reinforcement Learning agents are expected to eventually perform well. Typically, this takes the form of a guarantee about the asymptotic behavior of an algorithm given some assumptions about the environment. We present an algorithm for a policy whose value approaches the optimal value with probability 1 in all computable probabilistic environments, provided the agent has a bounded horizon. This is known as strong asymptotic optimality, and it was previously unknown whether it was possible for a policy to be strongly asymptotically optimal in the class of all computable probabilistic environments. Our agent, Inquisitive Reinforcement Learner (Inq), is more likely to explore the more it expects an exploratory action to reduce its uncertainty about which environment it is in, hence the term inquisitive. Exploring inquisitively is a strategy that can be applied generally; for more manageable environment classes, inquisitiveness is tractable. We conducted experiments in "grid-worlds" to compare the Inquisitive Reinforcement Learner to other weakly asymptotically optimal agents.

📄 PDF Abstract BibTeX arXiv:1903.01021

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

A Model-Free Universal AI

2026-02-26 · Yegon Kim, Juho Lee arxiv

In general reinforcement learning, all established optimal agents, including AIXI, are model-based, explicitly maintaining and using environment models. This paper introduces Universal AI with Q-Induction (AIQI), the fir…

General Reinforcement Learning

Thompson Sampling is Asymptotically Optimal in General Environments

2016-02-25 · Jan Leike, Tor Lattimore, Laurent Orseau, Marcus Hutter

We discuss a variant of Thompson sampling for nonparametric reinforcement learning in a countable classes of general stochastic environments. These environments can be non-Markov, non-ergodic, and partially observable. W…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

Nonparametric General Reinforcement Learning

2016-11-28 · Jan Leike

Reinforcement learning (RL) problems are often phrased in terms of Markov decision processes (MDPs). In this thesis we go beyond MDPs and consider RL in environments that are non-Markovian, non-ergodic and only partially…

General Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Curiosity Killed or Incapacitated the Cat and the Asymptotically Optimal Agent

2020-06-05 · Michael K. Cohen, Elliot Catt, Marcus Hutter

Reinforcement learners are agents that learn to pick actions that lead to high reward. Ideally, the value of a reinforcement learner's policy approaches optimality--where the optimal informed policy is the one which maxi…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration

Distributed stochastic optimization with gradient tracking over strongly-connected networks

2019-03-18 · Ran Xin, Anit Kumar Sahu, Usman A. Khan, Soummya Kar

In this paper, we study distributed stochastic optimization to minimize a sum of smooth and strongly-convex local cost functions over a network of agents, communicating over a strongly-connected graph. Assuming that each…

Stochastic Optimization