Multi-Object Manipulation via Object-Centric Neural Scattering Functions
Learned visual dynamics models have proven effective for robotic manipulation tasks. Yet, it remains unclear how best to represent scenes involving multi-object interactions. Current methods decompose a scene into discrete objects, but they struggle with precise modeling and manipulation amid challenging lighting conditions as they only encode appearance tied with specific illuminations. In this work, we propose using object-centric neural scattering functions (OSFs) as object representations in a model-predictive control framework. OSFs model per-object light transport, enabling compositional scene re-rendering under object rearrangement and varying lighting conditions. By combining this approach with inverse parameter estimation and graph-based neural dynamics models, we demonstrate improved model-predictive control performance and generalization in compositional multi-object environments, even in previously unseen scenarios and harsh lighting conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Model Predictive ControlObjectObject Rearrangementparameter estimationSimilar Papers 제목 키워드 기반
Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Composition
Photorealistic object appearance modeling from 2D images is a constant topic in vision and graphics. While neural implicit methods (such as Neural Radiance Fields) have shown high-fidelity view synthesis results, they ca…
Inverse RenderingObjectDisentangled Object-Centric Image Representation for Robotic Manipulation
Learning robotic manipulation skills from vision is a promising approach for developing robotics applications that can generalize broadly to real-world scenarios. As such, many approaches to enable this vision have been …
ObjectSlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation
Inspired by how humans reason over discrete objects and their relationships, we explore whether compact object-centric and object-relation representations can form a foundation for multitask robotic manipulation. Most ex…
FOCUS: Object-Centric World Models for Robotics Manipulation
Understanding the world in terms of objects and the possible interplays with them is an important cognition ability, especially in robotics manipulation, where many tasks require robot-object interactions. However, learn…
ObjectLearning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Manipulation tasks can often be decomposed into multiple subtasks performed in parallel, e.g., sliding an object to a goal pose while maintaining contact with a table. Individual subtasks can be achieved by task-axis con…
Objectreinforcement-learningReinforcement Learning (RL)Zero-shot Generalization