Behavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RL

2021-06-16 · Catherine Cang, Aravind Rajeswaran, Pieter Abbeel, Michael Laskin

Offline Reinforcement Learning (RL) aims to extract near-optimal policies from imperfect offline data without additional environment interactions. Extracting policies from diverse offline datasets has the potential to expand the range of applicability of RL by making the training process safer, faster, and more streamlined. We investigate how to improve the performance of offline RL algorithms, its robustness to the quality of offline data, as well as its generalization capabilities. To this end, we introduce Offline Model-based RL with Adaptive Behavioral Priors (MABE). Our algorithm is based on the finding that dynamics models, which support within-domain generalization, and behavioral priors, which support cross-domain generalization, are complementary. When combined together, they substantially improve the performance and generalization of offline RL policies. In the widely studied D4RL offline RL benchmark, we find that MABE achieves higher average performance compared to prior model-free and model-based algorithms. In experiments that require cross-domain generalization, we find that MABE outperforms prior methods. Our website is available at https://sites.google.com/berkeley.edu/mabe .

📄 PDF Abstract BibTeX arXiv:2106.09119

Code (0)

등록된 구현이 없습니다.

Tasks

D4RL Domain Generalization Offline RL Reinforcement Learning (RL)

Large Behavior Model: A Promptable Digital Twin of the Retail Customer

2026-07-08 · Wachiravit Modecrua, Krittin Pachtrachai, Touchapon Kraisingkorn arxiv

Customer behavior modeling underpins recommendation, marketing, and decision support, yet existing approaches either optimize predictive accuracy without explaining decisions or simulate users without grounding them in r…

Reinforcement Learning Decision Making

Learning Physics Priors for Deep Reinforcement Learing

2018-09-27 · Yilun Du, Karthik Narasimhan

While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is challenging and often requires substantial interactions with the env…

Deep Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

2022-06-23 · Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga 외

Pretraining on noisy, internet-scale datasets has been heavily studied as a technique for training models with broad, general capabilities for text, images, and other modalities. However, for many sequential decision dom…

Imitation Learning Minecraft reinforcement-learning Reinforcement Learning (RL)

Task-Agnostic Dynamics Priors for Deep Reinforcement Learning

2019-05-13 · Yilun Du, Karthik Narasimhan

While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is often challenging and requires substantial interaction with the envi…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Transferable Dynamics Priors from Action to World Modeling

2026-06-28 · Ze Huang, Jiahui Zhang, Hairuo Liu, Chenxi Zhang 외 arxiv

We study action-conditioned world modeling as a scalable way to learn transferable dynamics priors for robot learning. By pretraining a model to predict how actions drive visual scene evolution, the resulting world model…

Robot Manipulation Video Generation