Objects as Spatio-Temporal 2.5D points
Determining accurate bird's eye view (BEV) positions of objects and tracks in a scene is vital for various perception tasks including object interactions mapping, scenario extraction etc., however, the level of supervision required to accomplish that is extremely challenging to procure. We propose a light-weight, weakly supervised method to estimate 3D position of objects by jointly learning to regress the 2D object detections and scene's depth prediction in a single feed-forward pass of a network. Our proposed method extends a center-point based single-shot object detector, and introduces a novel object representation where each object is modeled as a BEV point spatio-temporally, without the need of any 3D or BEV annotations for training and LiDAR data at query time. The approach leverages readily available 2D object supervision along with LiDAR point clouds (used only during training) to jointly train a single network, that learns to predict 2D object detection alongside the whole scene's depth, to spatio-temporally model object tracks as points in BEV. The proposed method is computationally over $\sim$10x efficient compared to recent SOTA approaches while achieving comparable accuracies on KITTI tracking benchmark.
Code (0)
등록된 구현이 없습니다.
Tasks
2D Object DetectionDepth EstimationDepth PredictionObjectobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
SpOT: Spatiotemporal Modeling for 3D Object Tracking
3D multi-object tracking aims to uniquely and consistently identify all mobile entities through time. Despite the rich spatiotemporal information available in this setting, current 3D tracking methods primarily rely on a…
3D Multi-Object Tracking3D Object TrackingMulti-Object TrackingObject+1Intensity Video Guided 4D Fusion for Improved Highly Dynamic 3D Reconstruction
The availability of high-speed 3D video sensors has greatly facilitated 3D shape acquisition of dynamic and deformable objects, but high frame rate 3D reconstruction is always degraded by spatial noise and temporal fluct…
3D ReconstructionModeling Complex Object Changes in Satellite Image Time-Series: Approach based on CSP and Spatiotemporal Graph
This paper proposes a method for automatically monitoring and analyzing the evolution of complex geographic objects. The objects are modeled as a spatiotemporal graph, which separates filiation relations, spatial relatio…
Time SeriesDropletVideo: A Dataset and Approach to Explore Integral Spatio-Temporal Consistent Video Generation
Spatio-temporal consistency is a critical research topic in video generation. A qualified generated video segment must ensure plot plausibility and coherence while maintaining visual consistency of objects and scenes acr…
Video GenerationVIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation
Referring Video Object Segmentation (RVOS) aims to segment target objects in videos based on natural language descriptions. However, fixed keyframe-based approaches that couple a vision language model with a separate pro…
Referring Video Object Segmentation