Single Level Feature-to-Feature Forecasting with Deformable Convolutions
Future anticipation is of vital importance in autonomous driving and other decision-making systems. We present a method to anticipate semantic segmentation of future frames in driving scenarios based on feature-to-feature forecasting. Our method is based on a semantic segmentation model without lateral connections within the upsampling path. Such design ensures that the forecasting addresses only the most abstract features on a very coarse resolution. We further propose to express feature-to-feature forecasting with deformable convolutions. This increases the modelling power due to being able to represent different motion patterns within a single feature map. Experiments show that our models with deformable convolutions outperform their regular and dilated counterparts while minimally increasing the number of parameters. Our method achieves state of the art performance on the Cityscapes validation set when forecasting nine timesteps into the future.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingDecision MakingSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motion
Dense semantic forecasting anticipates future events in video by inferring pixel-level semantics of an unobserved future image. We present a novel approach that is applicable to various single-frame architectures and tas…
Future predictionPanoptic SegmentationregressionSegmentation+1Snipper: A Spatiotemporal Transformer for Simultaneous Multi-Person 3D Pose Estimation Tracking and Forecasting on a Video Snippet
Multi-person pose understanding from RGB videos involves three complex tasks: pose estimation, tracking and motion forecasting. Intuitively, accurate multi-person pose estimation facilitates robust tracking, and robust t…
3D Pose EstimationMotion ForecastingMulti-Person Pose EstimationPose EstimationSEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images
Learning deformable 3D object models from single-view in-the-wild images has enabled impressive 3D shape reconstruction without supervision. However, it remains unclear whether these models capture the semantic structure…
3D Shape ReconstructionSemantic correspondenceTADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection
Most single-stage 3D object detectors complete different tasks with the same extracted features. Nevertheless, it is impossible to project features into a common space that is adaptive for all the tasks. We present a nov…
3D Object DetectionAirport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approach
Accurate forecasting of passenger flows is critical for maintaining the efficiency and resilience of airport operations. Recent advances in patch-based Transformer models have shown strong potential in various time serie…
Time Series Forecasting