Planning Robot Motion using Deep Visual Prediction
In this paper, we introduce a novel framework that can learn to make visual predictions about the motion of a robotic agent from raw video frames. Our proposed motion prediction network (PROM-Net) can learn in a completely unsupervised manner and efficiently predict up to 10 frames in the future. Moreover, unlike any other motion prediction models, it is lightweight and once trained it can be easily implemented on mobile platforms that have very limited computing capabilities. We have created a new robotic data set comprising LEGO Mindstorms moving along various trajectories in three different environments under different lighting conditions for testing and training the network. Finally, we introduce a framework that would use the predicted frames from the network as an input to a model predictive controller for motion planning in unknown dynamic environments with moving obstacles.
Code (0)
등록된 구현이 없습니다.
Tasks
Motion Planningmotion predictionPredictionSimilar Papers 제목 키워드 기반
Material Recognition CNNs and Hierarchical Planning for Biped Robot Locomotion on Slippery Terrain
In this paper we tackle the problem of visually predicting surface friction for environments with diverse surfaces, and integrating this knowledge into biped robot locomotion planning. The problem is essential for autono…
FrictionMaterial RecognitionH-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
World models are becoming central to robotic planning and control as they enable prediction of future state transitions. Existing approaches often emphasize video generation or natural-language prediction, which are diff…
Visual GroundingVideo GenerationMotion PlanningCounterfactual Reasoning about Intent for Interactive Navigation in Dynamic Environments
Many modern robotics applications require robots to function autonomously in dynamic environments including other decision making agents, such as people or other robots. This calls for fast and scalable interactive motio…
counterfactualCounterfactual ReasoningDecision MakingMotion Planning+2Accelerating Integrated Task and Motion Planning with Neural Feasibility Checking
As robots play an increasingly important role in the industrial, the expectations about their applications for everyday living tasks are getting higher. Robots need to perform long-horizon tasks that consist of several s…
Motion PlanningTask and Motion PlanningERUPT: An Open Toolkit for Interfacing with Robot Motion Planners in Extended Reality
We propose the Extended Reality Universal Planning Toolkit (ERUPT), an extended reality (XR) system for interactive motion planning. Our system allows users to create and dynamically reconfigure environments while they p…
Motion Planning