Fast exploration and learning of latent graphs with aliased observations

2023-03-13 · Miguel Lazaro-Gredilla, Ishan Deshpande, Sivaramakrishnan Swaminathan, Meet Dave, Dileep George

We consider the problem of recovering a latent graph where the observations at each node are \emph{aliased}, and transitions are stochastic. Observations are gathered by an agent traversing the graph. Aliasing means that multiple nodes emit the same observation, so the agent can not know in which node it is located. The agent needs to uncover the hidden topology as accurately as possible and in as few steps as possible. This is equivalent to efficient recovery of the transition probabilities of a partially observable Markov decision process (POMDP) in which the observation probabilities are known. An algorithm for efficiently exploring (and ultimately recovering) the latent graph is provided. Our approach is exponentially faster than naive exploration in a variety of challenging topologies with aliased observations while remaining competitive with existing baselines in the unaliased regime.

📄 PDF Abstract BibTeX arXiv:2303.07397

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient Exploration

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Differentiable Clone-Structured Causal Graphs for End-to-End Cognitive Map Learning from Image Sequences

2026-07-14 · Arash Nikzad, Sasan Sarbishegi, Ali Dasmeh, Muhammad Asif 외 arxiv

How can an agent build a structured map of its world from nothing but an ongoing sequence of raw sensory input and its own movements, especially when natural variation means exact sensory patterns rarely repeat? The Clon…

Graph schemas as abstractions for transfer learning, inference, and planning

2023-02-14 · J. Swaroop Guntupalli, Rajkumar Vasudeva Raju, Shrinu Kushagra, Carter Wendelken 외

Transferring latent structure from one environment or problem to another is a mechanism by which humans and animals generalize with very little data. Inspired by cognitive and neurobiological insights, we propose graph s…

Graph Learning Hippocampus Transfer Learning

Provably efficient RL with Rich Observations via Latent State Decoding

2019-01-25 · Simon S. Du, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal 외

We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states. Under certain identifiability assumptions, we demonstrate how to estimate a mapping from the observ…

Clustering Q-Learning regression

State-Wise Safe Reinforcement Learning With Pixel Observations

2023-11-03 · Simon Sinong Zhan, YiXuan Wang, Qingyuan Wu, Ruochen Jiao 외

In the context of safe exploration, Reinforcement Learning (RL) has long grappled with the challenges of balancing the tradeoff between maximizing rewards and minimizing safety violations, particularly in complex environ…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration+1

Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments

2024-01-11 · Antoine Dedieu, Wolfgang Lehrach, Guangyao Zhou, Dileep George 외

Despite their stellar performance on a wide range of tasks, including in-context tasks only revealed during inference, vanilla transformers and variants trained for next-token predictions (a) do not learn an explicit wor…