paper-with-me

홈 › Papers

PhyPlan: Compositional and Adaptive Physical Task Reasoning with Physics-Informed Skill Networks for Robot Manipulators

2024-02-24 · Harshil Vagadia, Mudit Chopra, Abhinav Barnawal, Tamajit Banerjee, Shreshth Tuli, Souvik Chakraborty, Rohan Paul

Given the task of positioning a ball-like object to a goal region beyond direct reach, humans can often throw, slide, or rebound objects against the wall to attain the goal. However, enabling robots to reason similarly is non-trivial. Existing methods for physical reasoning are data-hungry and struggle with complexity and uncertainty inherent in the real world. This paper presents PhyPlan, a novel physics-informed planning framework that combines physics-informed neural networks (PINNs) with modified Monte Carlo Tree Search (MCTS) to enable embodied agents to perform dynamic physical tasks. PhyPlan leverages PINNs to simulate and predict outcomes of actions in a fast and accurate manner and uses MCTS for planning. It dynamically determines whether to consult a PINN-based simulator (coarse but fast) or engage directly with the actual environment (fine but slow) to determine optimal policy. Evaluation with robots in simulated 3D environments demonstrates the ability of our approach to solve 3D-physical reasoning tasks involving the composition of dynamic skills. Quantitatively, PhyPlan excels in several aspects: (i) it achieves lower regret when learning novel tasks compared to state-of-the-art, (ii) it expedites skill learning and enhances the speed of physical reasoning, (iii) it demonstrates higher data efficiency compared to a physics un-informed approach.

📄 PDF Abstract BibTeX arXiv:2402.15767

Code (1)

phyplan/phyplan 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Similar Papers 제목 키워드 기반

PhyRPR: Training-Free Physics-Constrained Video Generation

2026-01-14 · Yibo Zhao, Hengjia Li, Xiaofei He, Boxi Wu arxiv

Recent diffusion-based video generation models can synthesize visually plausible videos, yet they often struggle to satisfy physical constraints. A key reason is that most existing approaches remain single-stage: they en…

Video Generation

ComPhy: Compositional Physical Reasoning of Objects and Events from Videos

2022-05-02 · ICLR 2022 4 · Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding 외

Objects' motions in nature are governed by complex interactions and their properties. While some properties, such as shape and material, can be identified via the object's visual appearances, others like mass and electri…

Compositional Physical Reasoning of Objects and Events from Videos

2024-08-02 · Zhenfang Chen, Shilong Dong, Kexin Yi, Yunzhu Li 외

Understanding and reasoning about objects' physical properties in the natural world is a fundamental challenge in artificial intelligence. While some properties like colors and shapes can be directly observed, others, su…

counterfactualQuestion Answering

CounterCurate: Enhancing Physical and Semantic Visio-Linguistic Compositional Reasoning via Counterfactual Examples

2024-02-20 · Jianrui Zhang, Mu Cai, Tengyang Xie, Yong Jae Lee

We propose CounterCurate, a framework to comprehensively improve the visio-linguistic compositional reasoning capability for both contrastive and generative multimodal models. In particular, we identify two critical unde…

counterfactualData AugmentationImage GenerationText Generation

Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding

2023-01-22 · Juncheng Li, Siliang Tang, Linchao Zhu, Wenqiao Zhang 외

Temporal grounding is the task of locating a specific segment from an untrimmed video according to a query sentence. This task has achieved significant momentum in the computer vision community as it enables activity gro…

DiversitySemantic correspondenceSentence