DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation
Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/
Code (3)
Tasks
Robot ManipulationSimilar Papers 제목 키워드 기반
Bridging the Physics-Data Gap with FNO-Guided Conditional Flow Matching: Designing Inductive Bias through Hierarchical Physical Constraints
Conventional time-series generation often ignores domain-specific physical constraints, limiting statistical and physical consistency. We propose a hierarchical framework that embeds the inherent hierarchy of physical la…
Human Activity RecognitionHPG-Diff: Hierarchical physics-guided diffusion with differentiable connectivity constraints for topology optimization
Deep generative models offer a promising paradigm for topology optimization, enabling rapid design exploration. However, these approaches lack intrinsic physics guidance, often leading to poor generalizability across uns…
Hierarchical clustering in particle physics through reinforcement learning
Particle physics experiments often require the reconstruction of decay patterns through a hierarchical clustering of the observed final-state particles. We show that this task can be phrased as a Markov Decision Process …
Clusteringreinforcement-learningReinforcement LearningReinforcement Learning (RL)Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Control
We present Diffuse-CLoC, a guided diffusion framework for physics-based look-ahead control that enables intuitive, steerable, and physically realistic motion generation. While existing kinematics motion generation with d…
Action GenerationMotion Generationmotion in-betweeningPhysics-Guided Hierarchical Reward Mechanism for Learning-Based Robotic Grasping
Learning-based grasping can afford real-time grasp motion planning of multi-fingered robotics hands thanks to its high computational efficiency. However, learning-based methods are required to explore large search spaces…
Computational EfficiencyDeep Reinforcement LearningMotion Planningreinforcement-learning+3