Flow-Enabled Generalization to Human Demonstrations in Few-Shot Imitation Learning

2026-02-11 · Runze Tang, Penny Sweetser arxiv

Imitation Learning (IL) enables robots to learn complex skills from demonstrations without explicit task modeling, but it typically requires large amounts of demonstrations, creating significant collection costs. Prior work has investigated using flow as an intermediate representation to enable the use of human videos as a substitute, thereby reducing the amount of required robot demonstrations. However, most prior work has focused on the flow, either on the object or on specific points of the robot/hand, which cannot describe the motion of interaction. Meanwhile, relying on flow to achieve generalization to scenarios observed only in human videos remains limited, as flow alone cannot capture precise motion details. Furthermore, conditioning on scene observation to produce precise actions may cause the flow-conditioned policy to overfit to training tasks and weaken the generalization indicated by the flow. To address these gaps, we propose SFCrP, which includes a Scene Flow prediction model for Cross-embodiment learning (SFCr) and a Flow and Cropped point cloud conditioned Policy (FCrP). SFCr learns from both robot and human videos and predicts any point trajectories. FCrP follows the general flow motion and adjusts the action based on observations for precision tasks. Our method outperforms SOTA baselines across various real-world task settings, while also exhibiting strong spatial and instance generalization to scenarios seen only in human videos.

📄 PDF Abstract BibTeX arXiv:2602.10594

Code (0)

등록된 구현이 없습니다.

Building Explicit World Model for Zero-Shot Open-World Object Manipulation

2026-03-14 · Xiaotong Li, Gang Chen, Javier Alonso-Mora arxiv

Open-world object manipulation remains a fundamental challenge in robotics. While Vision-Language-Action (VLA) models have demonstrated promising results, they rely heavily on large-scale robot action demonstrations, whi…

Zero-shot Generalization

Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models

2026-02-13 · Nick Heppert, Minh Quang Nguyen, Abhinav Valada arxiv

Imitation learning is a popular paradigm to teach robots new tasks, but collecting robot demonstrations through teleoperation or kinesthetic teaching is tedious and time-consuming. In contrast, directly demonstrating a t…

AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations

2024-11-20 · Gaurav Verma, Rachneet Kaur, Nishan Srishankar, Zhen Zeng 외

State-of-the-art multimodal web agents, powered by Multimodal Large Language Models (MLLMs), can autonomously execute many web tasks by processing user instructions and interacting with graphical user interfaces (GUIs). …

Few-Shot Learning Meta-Learning

AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning

2026-08-14 · Zhiyue Zhao, Jingyi Wu, Hairuo Liu, Mingyu Liu 외 arxiv

Dexterous manipulation is a fundamental capability for embodied intelligence, but scaling it remains difficult because robot demonstrations are expensive to collect and action spaces vary across embodiments. Policies tra…

Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning

2020-11-01 · ICLR 2021 1 · Valerie Chen, Abhinav Gupta, Kenneth Marino

Complex, multi-task problems have proven to be difficult to solve efficiently in a sparse-reward reinforcement learning setting. In order to be sample efficient, multi-task learning requires reuse and sharing of low-leve…

Multi-Task Learning reinforcement-learning Reinforcement Learning (RL)