AgenticFocus: Object-Preserving Mixed Reality Synthesis from Human FPV Video for Dexterous Humanoid Learning
Human egocentric video is a scalable supervision source for humanoid policy learning, but current pipelines struggle with hand-object occlusion, oversimplified motion, or specialized capture hardware. We introduce AgenticFocus, a Mixed Reality synthesis pipeline that converts ordinary first-person-view human videos into robot-trainable demonstrations by restoring occluded object geometry, reconstructing full-hand motion, and retargeting it to a humanoid embodiment through camera-relative alignment and layered compositing. The resulting dataset pairs focused visual observations with synchronized robot actions and states. AgenticFocus achieves lower trajectory error and smoother wrist motion than cross-embodiment baselines, with SPARC scores of -5.18 versus -5.56 and -6.05.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Mutual Scene Synthesis for Mixed Reality Telepresence
Remote telepresence via next-generation mixed reality platforms can provide higher levels of immersion for computer-mediated communications, allowing participants to engage in a wide spectrum of activities, previously no…
Mixed RealityMulti-layer Visualization for Medical Mixed Reality
Medical Mixed Reality helps surgeons to contextualize intraoperative data with video of the surgical scene. Nonetheless, the surgical scene and anatomical target are often occluded by surgical instruments and surgeon han…
Mixed RealityA Real-Time Diminished Reality Approach to Privacy in MR Collaboration
Diminished reality (DR) refers to the digital removal of real-world objects by compositing background content in their place. This thesis presents a real-time, inpainting-based DR system designed to enable privacy contro…
Semantic SegmentationObject DetectionVideo InpaintingLLMR: Real-time Prompting of Interactive Worlds using Large Language Models
We present Large Language Model for Mixed Reality (LLMR), a framework for the real-time creation and modification of interactive Mixed Reality experiences using LLMs. LLMR leverages novel strategies to tackle difficult c…
Language ModelingLanguage ModellingLarge Language ModelManagement+4Point Cloud Registration of non-rigid objects in sparse 3D Scans with applications in Mixed Reality
Point Cloud Registration is the problem of aligning the corresponding points of two 3D point clouds referring to the same object. The challenges include dealing with noise and partial match of real-world 3D scans. For no…
Mixed RealityObjectPoint Cloud Registration