GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulation
Effective robotic manipulation relies on a precise understanding of 3D scene geometry, and one of the most straightforward ways to acquire such geometry is through multi-view observations. Motivated by this, we present GP3 -- a 3D geometry-aware robotic manipulation policy that leverages multi-view input. GP3 employs a spatial encoder to infer dense spatial features from RGB observations, which enable the estimation of depth and camera parameters, leading to a compact yet expressive 3D scene representation tailored for manipulation. This representation is fused with language instructions and translated into continuous actions via a lightweight policy head. Comprehensive experiments demonstrate that GP3 consistently outperforms state-of-the-art methods on simulated benchmarks. Furthermore, GP3 transfers effectively to real-world robots without depth sensors or pre-mapped environments, requiring only minimal fine-tuning. These results highlight GP3 as a practical, sensor-agnostic solution for geometry-aware robotic manipulation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images
Novel view synthesis aims to generate new view images of a given view image collection. Recent attempts address this problem relying on 3D geometry priors (e.g., shapes, sizes, and positions) learned from multi-view imag…
3D geometryNeRFNovel View SynthesisGeometry-Aware Reference Synthesis for Multi-View Image Super-Resolution
Recent multi-view multimedia applications struggle between high-resolution (HR) visual experience and storage or bandwidth constraints. Therefore, this paper proposes a Multi-View Image Super-Resolution (MVISR) task. It …
Image Super-ResolutionSuper-ResolutionVideo Super-ResolutionMulti-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis
3D-aware image synthesis aims to generate images of objects from multiple views by learning a 3D representation. However, one key challenge remains: existing approaches lack geometry constraints, hence usually fail to ge…
3D-Aware Image Synthesis3D geometryImage GenerationImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detection
We propose ImGeoNet, a multi-view image-based 3D object detection framework that models a 3D space by an image-induced geometry-aware voxel representation. Unlike previous methods which aggregate 2D features into 3D voxe…
3D Object Detectionobject-detectionObject DetectionMoGaFace: Momentum-Guided and Texture-Aware Gaussian Avatars for Consistent Facial Geometry
Existing 3D head avatar reconstruction methods adopt a two-stage process, relying on tracked FLAME meshes derived from facial landmarks, followed by Gaussian-based rendering. However, misalignment between the estimated m…