Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness
Diffusion policies excel at visuomotor control but often fail catastrophically under severe out-of-distribution (OOD) disturbances, such as unexpected object displacements or visual corruptions. To address this vulnerability, we introduce the Dream Diffusion Policy (DDP), a framework that deeply integrates a diffusion world model into the policy's training objective via a shared 3D visual encoder. This co-optimization endows the policy with robust state-prediction capabilities. When encountering sudden OOD anomalies during inference, DDP detects the real-imagination discrepancy and actively abandons the corrupted visual stream. Instead, it relies on its internal "imagination" (autoregressively forecasted latent dynamics) to safely bypass the disruption, generating imagined trajectories before smoothly realigning with physical reality. Extensive evaluations demonstrate DDP's exceptional resilience. Notably, DDP achieves a 73.8% OOD success rate on MetaWorld (vs. 23.9% without predictive imagination) and an 83.3% success rate under severe real-world spatial shifts (vs. 3.3% without predictive imagination). Furthermore, as a stress test, DDP maintains a 76.7% real-world success rate even when relying entirely on open-loop imagination post-initialization.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning Real-World Robot Policies by Dreaming
Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Towards biologically plausible Dreaming and Planning in recurrent spiking networks
Humans and animals can learn new skills after practicing for a few hours, while current reinforcement learning algorithms require a large amount of data to achieve good performances. Recent model-based approaches show pr…
Autonomous DrivingModel-based Reinforcement LearningOpen-Ended Question AnsweringReinforcement Learning (RL)Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization
Zero-shot generalization (ZSG) to unseen dynamics is a major challenge for creating generally capable embodied agents. To address the broader challenge, we start with the simpler setting of contextual reinforcement learn…
Zero-shot GeneralizationDreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pix…
Contrastive LearningModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1Forward KL Regularized Preference Optimization for Aligning Diffusion Policies
Diffusion models have achieved remarkable success in sequential decision-making by leveraging the highly expressive model capabilities in policy learning. A central problem for learning diffusion policies is to align the…
D4RLDecision MakingReinforcement Learning (RL)Sequential Decision Making