A Reliable Representation with Bidirectional Transition Model for Visual Reinforcement Learning Generalization

2023-12-04 · Xiaobo Hu, Youfang Lin, Yue Liu, Jinwen Wang, Shuo Wang, Hehe Fan, Kai Lv

Visual reinforcement learning has proven effective in solving control tasks with high-dimensional observations. However, extracting reliable and generalizable representations from vision-based observations remains a central challenge. Inspired by the human thought process, when the representation extracted from the observation can predict the future and trace history, the representation is reliable and accurate in comprehending the environment. Based on this concept, we introduce a Bidirectional Transition (BiT) model, which leverages the ability to bidirectionally predict environmental transitions both forward and backward to extract reliable representations. Our model demonstrates competitive generalization performance and sample efficiency on two settings of the DeepMind Control suite. Additionally, we utilize robotic manipulation and CARLA simulators to demonstrate the wide applicability of our method.

📄 PDF Abstract BibTeX arXiv:2312.01915

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

CARLA CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban…

Domain Adaptation with Adaptive Imagination for Visual Reinforcement Learning under Limited Target Data

2026-06-29 · Hyunwoo Park, Sang-Hyun Lee arxiv

Sim-to-real transfer remains a major obstacle for reinforcement learning (RL), especially for vision-based control where image observations exacerbate the state-distribution shift between simulation and the real world. D…

Reinforcement Learning Domain Adaptation

Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination

2022-06-16 · Jiafei Lyu, Xiu Li, Zongqing Lu

The learned policy of model-free offline reinforcement learning (RL) methods is often constrained to stay within the support of datasets to avoid possible dangerous out-of-distribution actions or states, making it challe…

D4RL Offline RL Reinforcement Learning (RL)

Understanding Self-Predictive Learning for Reinforcement Learning

2022-12-06 · Yunhao Tang, Zhaohan Daniel Guo, Pierre Harvey Richemond, Bernardo Ávila Pires 외

We study the learning dynamics of self-predictive learning for reinforcement learning, a family of algorithms that learn representations by minimizing the prediction error of their own future latent representations. Desp…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning

2026-06-10 · Chao Lei, Yanbei Jiang, Markus Hiller, Zhijian Zhou 외 arxiv

Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediat…

Reinforcement Learning Spatial Reasoning Visual Reasoning

Generalization in Visual Reinforcement Learning with the Reward Sequence Distribution

2023-02-19 · Jie Wang, Rui Yang, Zijie Geng, Zhihao Shi 외

Generalization in partially observed markov decision processes (POMDPs) is critical for successful applications of visual reinforcement learning (VRL) in real scenarios. A widely used idea is to learn task-relevant repre…