Learning Neural Implicit Functions as Object Representations for Robotic Manipulation
Robotic manipulation planning is the problem of finding a sequence of robot configurations that involves interactions with objects in the scene, e.g., grasp, placement, tool-use, etc. To achieve such interactions, traditional approaches require hand-designed features and object representations, and it still remains an open question how to describe such interactions with arbitrary objects in a flexible and efficient way. Inspired by neural implicit representations in 3D modeling, e.g. NeRF, we propose a method to represent objects as neural implicit functions upon which we can define and jointly train interaction features. The proposed pixel-aligned representation is directly inferred from camera images with known camera geometry, naturally acting as a perception component in the whole manipulation pipeline, while at the same time enabling sequential robot manipulation planning.
Code (0)
등록된 구현이 없습니다.
Tasks
NeRFOpen-Ended Question AnsweringRobot ManipulationSimilar Papers 제목 키워드 기반
Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering
A significant challenge for real-world robotic manipulation is the effective 6DoF grasping of objects in cluttered scenes from any single viewpoint without the need for additional scene exploration. This work reinterpret…
Grasp GenerationRobotic GraspingLearning Part Motion of Articulated Objects Using Spatially Continuous Neural Implicit Representations
Articulated objects (e.g., doors and drawers) exist everywhere in our life. Different from rigid objects, articulated objects have higher degrees of freedom and are rich in geometries, semantics, and part functions. Mode…
KUDA: Keypoints to Unify Dynamics Learning and Visual Prompting for Open-Vocabulary Robotic Manipulation
With the rapid advancement of large language models (LLMs) and vision-language models (VLMs), significant progress has been made in developing open-vocabulary robotic manipulation systems. However, many existing approach…
ObjectVisual PromptingMulti-Object Manipulation via Object-Centric Neural Scattering Functions
Learned visual dynamics models have proven effective for robotic manipulation tasks. Yet, it remains unclear how best to represent scenes involving multi-object interactions. Current methods decompose a scene into discre…
Model Predictive ControlObjectObject Rearrangementparameter estimationRethinking Implicit Spatial Representation in Visuomotor Policy Learning
Generative model-based imitation learning has become a widely adopted paradigm for robotic manipulation, where policy performance depends critically on the conditioned visual representations. Although spatial softmax-bas…