Discrete Predictive Representation for Long-horizon Planning
Discrete representations have been key in enabling robots to plan at more abstract levels and solve temporally-extended tasks more efficiently for decades. However, they typically require expert specifications. On the other hand, deep reinforcement learning aims to learn to solve tasks end-to-end, but struggles with long-horizon tasks. In this work, we propose Discrete Object-factorized Representation Planning (DORP), which learns temporally-abstracted discrete representations from exploratory video data in an unsupervised fashion via a mutual information maximization objective. DORP plans a sequence of abstract states for a low-level model-predictive controller to follow. In our experiments, we show that DORP robustly solves unseen long-horizon tasks. Interestingly, it discovers independent representations per object and binary properties such as a key-and-door.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningObjectReinforcement Learning (RL)Similar Papers 제목 키워드 기반
QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learning
Inspired by the success of the Transformer architecture in natural language processing and computer vision, we investigate the use of Transformers in Reinforcement Learning (RL), specifically in modeling the environment'…
continuous-controlContinuous ControlModel Predictive ControlQ-Learning+1Diffusion-MPC in Discrete Domains: Feasibility Constraints, Horizon Effects, and Critic Alignment: Case study with Tetris
We study diffusion-based model predictive control (Diffusion-MPC) in discrete combinatorial domains using Tetris as a case study. Our planner samples candidate placement sequences with a MaskGIT-style discrete denoiser a…
Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning
Recently, world models have emerged as a promising paradigm for building intelligent agents by learning predictive models that estimate future environment states conditioned on observations and actions. In particular, JE…
Predictive but Not Plannable: RC-aux for Latent World Models
A latent world model may achieve accurate short-horizon prediction while still inducing a latent space that is poorly aligned with planning. A key issue is spatiotemporal mismatch: these models are often trained with loc…
From proprioception to long-horizon planning in novel environments: A hierarchical RL model
For an intelligent agent to flexibly and efficiently operate in complex environments, they must be able to reason at multiple levels of temporal, spatial, and conceptual abstraction. At the lower levels, the agent must i…
Efficient ExplorationModel Predictive ControlMuJoCo