Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation
We introduce Lucid-XR, a generative data engine for creating diverse and realistic-looking multi-modal data to train real-world robotic systems. At the core of Lucid-XR is vuer, a web-based physics simulation environment that runs directly on the XR headset, enabling internet-scale access to immersive, latency-free virtual interactions without requiring specialized equipment. The complete system integrates on-device physics simulation with human-to-robot pose retargeting. Data collected is further amplified by a physics-guided video generation pipeline steerable via natural language specifications. We demonstrate zero-shot transfer of robot visual policies to unseen, cluttered, and badly lit evaluation environments, after training entirely on Lucid-XR's synthetic data. We include examples across dexterous manipulation tasks that involve soft materials, loosely bound particles, and rigid body contact. Project website: https://lucidxr.github.io
Code (0)
등록된 구현이 없습니다.
Tasks
Video GenerationSimilar Papers 제목 키워드 기반
SAGE: Semantic-Driven Adaptive Gaussian Splatting in Extended Reality
3D Gaussian Splatting (3DGS) has significantly improved the efficiency and realism of three-dimensional scene visualization in several applications, ranging from robotics to eXtended Reality (XR). This work presents SAGE…
3DGSSemantic SegmentationAdaptive Invariant Extended Kalman Filter with Noise Covariance Tuning for Attitude Estimation
Attitude estimation is crucial in aerospace engineering, robotics, and virtual reality applications, but faces difficulties due to nonlinear system dynamics and sensor limitations. This paper addresses the challenge of a…
SplatBus: A Gaussian Splatting Viewer Framework via GPU Interprocess Communication
Radiance field-based rendering methods have attracted significant interest from the computer vision and computer graphics communities. They enable high-fidelity rendering with complex real-world lighting effects, but at …
Autonomous DrivingUnrealROX: An eXtremely Photorealistic Virtual Reality Environment for Robotics Simulations and Synthetic Data Generation
Data-driven algorithms have surpassed traditional techniques in almost every aspect in robotic vision problems. Such algorithms need vast amounts of quality data to be able to work properly after their training process. …
Depth Estimationobject-detectionObject DetectionSemantic Segmentation+1A Decade of Human-Robot Interaction Through Immersive Lenses: Reviewing Extended Reality as a Research Instrument in Social Robotics
Over the past decade, Extended Reality (XR), including Virtual, Augmented, and Mixed Reality, gained attention as a research instrument in human-robot interaction studies, but remains underexplored in empirical investiga…