Predicting Object Dynamics in Scenes
Given a static scene, a human can trivially enumerate the myriad of things that can happen next and characterize the relative likelihood of each. In the process, we make use of enormous amounts of commonsense knowledge about how the world works. In this paper, we investigate learning this commonsense knowledge from data. To overcome a lack of densely annotated spatiotemporal data, we learn from sequences of abstract images gathered using crowdsourcing. The abstract scenes provide both object location and attribute information. We demonstrate qualitatively and quantitatively that our models produce plausible scene predictions on both the abstract images, as well as natural images taken from the Internet.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeObjectSimilar Papers 제목 키워드 기반
Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning
Reliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other v…
Autonomous NavigationPositionSlotGNN: Unsupervised Discovery of Multi-Object Representations and Visual Dynamics
Learning multi-object dynamics from visual data using unsupervised techniques is challenging due to the need for robust, object representations that can be learned through robot interactions. This paper presents a novel …
ObjectObject DiscoveryObject RearrangementSpatial ReasoningReusable Slotwise Mechanisms
Agents with the ability to comprehend and reason about the dynamics of objects would be expected to exhibit improved robustness and generalization in novel scenarios. However, achieving this capability necessitates not o…
Future predictionObjectOut-of-Distribution GeneralizationQuestion Answering+1Zero-shot object prediction using semantic scene knowledge
This work focuses on the semantic relations between scenes and objects for visual object recognition. Semantic knowledge can be a powerful source of information especially in scenarios with few or no annotated training s…
ObjectObject RecognitionPredictionObjectForesight: Predicting Future 3D Object Trajectories from Human Videos
Humans can effortlessly anticipate how objects might move or change through interaction--imagining a cup being lifted, a knife slicing, or a lid being closed. We aim to endow computational systems with a similar ability …
3D Pose Estimation