Forecasting Future Instance Segmentation with Learned Optical Flow and Warping
For an autonomous vehicle it is essential to observe the ongoing dynamics of a scene and consequently predict imminent future scenarios to ensure safety to itself and others. This can be done using different sensors and modalities. In this paper we investigate the usage of optical flow for predicting future semantic segmentations. To do so we propose a model that forecasts flow fields autoregressively. Such predictions are then used to guide the inference of a learned warping function that moves instance segmentations on to future frames. Results on the Cityscapes dataset demonstrate the effectiveness of optical-flow methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Instance SegmentationOptical Flow EstimationSemantic SegmentationSimilar Papers 제목 키워드 기반
Predicting Future Instance Segmentation by Forecasting Convolutional Features
Anticipating future events is an important prerequisite towards intelligent behavior. Video forecasting has been studied as a proxy task towards this goal. Recent work has shown that to predict semantic segmentation of f…
Instance SegmentationOptical Flow EstimationSegmentationSemantic Segmentation+2HAMON: Passive Optical Sequence Mixing for Long-Horizon Forecasting
Simple linear and frequency-domain models remain surprisingly competitive in long-horizon time-series forecasting, and recent mechanistic evidence suggests that standard forecasting benchmarks may not require the dense s…
Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motion
Dense semantic forecasting anticipates future events in video by inferring pixel-level semantics of an unobserved future image. We present a novel approach that is applicable to various single-frame architectures and tas…
Future predictionPanoptic SegmentationregressionSegmentation+1Recurrent Flow-Guided Semantic Forecasting
Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key t…
Autonomous VehiclesOptical Flow EstimationPredictionVisuomotor Understanding for Representation Learning of Driving Scenes
Dashboard cameras capture a tremendous amount of driving scene video each day. These videos are purposefully coupled with vehicle sensing data, such as from the speedometer and inertial sensors, providing an additional s…
Optical Flow EstimationRepresentation LearningScene UnderstandingSemantic Segmentation