Zero-Shot Off-Policy Learning

2026-02-02 · Arip Asadulaev, Maksim Bobrin, Salem Lahlou, Dmitry Dylov, Fakhri Karray, Martin Takac arxiv

Off-policy learning methods seek to derive an optimal policy directly from a fixed dataset of prior interactions. This objective presents significant challenges, primarily due to the inherent distributional shift and value function overestimation bias. These issues become even more noticeable in zero-shot reinforcement learning, where an agent trained on reward-free data must adapt to new tasks at test time without additional training. In this work, we address the off-policy problem in a zero-shot setting by discovering a theoretical connection of successor measures to stationary density ratios. Using this insight, our algorithm can infer optimal importance sampling ratios, effectively performing a stationary distribution correction with an optimal policy for any task on the fly. We benchmark our method in motion tracking tasks on SMPL Humanoid, continuous control on ExoRL, and for the long-horizon OGBench tasks. Our technique seamlessly integrates into forward-backward representation frameworks and enables fast-adaptation to new tasks in a training-free regime. More broadly, this work bridges off-policy learning and zero-shot adaptation, offering benefits to both research areas.

📄 PDF Abstract BibTeX arXiv:2602.01962

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Continuous Control

Towards Robust Zero-Shot Reinforcement Learning

2025-10-17 · Kexin Zheng, Lauriane Teyssier, Yinan Zheng, Yu Luo 외 arxiv

The recent development of zero-shot reinforcement learning (RL) has opened a new avenue for learning pre-trained generalist policies that can adapt to arbitrary new tasks in a zero-shot manner. While the popular Forward-…

Representation Learning Reinforcement Learning

Zero-Shot Coordination in Ad Hoc Teams with Generalized Policy Improvement and Difference Rewards

2025-10-17 · Rupal Nigam, Niket Parikh, Hamid Osooli, Mikihisa Yuasa 외 arxiv

Real-world multi-agent systems may require ad hoc teaming, where an agent must coordinate with other previously unseen teammates to solve a task in a zero-shot manner. Prior work often either selects a pretrained policy …

Distributional Successor Features Enable Zero-Shot Policy Optimization

2024-03-10 · Chuning Zhu, Xinqi Wang, Tyler Han, Simon S. Du 외

Intelligent agents must be generalists, capable of quickly adapting to various tasks. In reinforcement learning (RL), model-based RL learns a dynamics model of the world, in principle enabling transfer to arbitrary rewar…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Federated reinforcement learning for robot motion planning with zero-shot generalization

2024-03-20 · Zhenyuan Yuan, Siyuan Xu, Minghui Zhu

This paper considers the problem of learning a control policy for robot motion planning with zero-shot generalization, i.e., no data collection and policy adaptation is needed when the learned policy is deployed in new e…

Motion Planning Zero-shot Generalization

DexGrasp-Zero: A Morphology-Aligned Policy for Zero-Shot Cross-Embodiment Dexterous Grasping

2026-03-17 · Yuliang Wu, Yanhan Lin, WengKit Lao, Yuhao Lin 외 arxiv

To meet the demands of increasingly diverse dexterous hand hardware, it is crucial to develop a policy that enables zero-shot cross-embodiment grasping without redundant re-learning. Cross-embodiment alignment is challen…