End-to-end Recurrent Multi-Object Tracking and Trajectory Prediction with Relational Reasoning
The majority of contemporary object-tracking approaches do not model interactions between objects. This contrasts with the fact that objects' paths are not independent: a cyclist might abruptly deviate from a previously planned trajectory in order to avoid colliding with a car. Building upon HART, a neural class-agnostic single-object tracker, we introduce a multi-object tracking method MOHART capable of relational reasoning. Importantly, the entire system, including the understanding of interactions and relations between objects, is class-agnostic and learned simultaneously in an end-to-end fashion. We explore a number of relational reasoning architectures and show that permutation-invariant models outperform non-permutation-invariant alternatives. We also find that architectures using a single permutation invariant operation like DeepSets, despite, in theory, being universal function approximators, are nonetheless outperformed by a more complex architecture based on multi-headed attention. The latter better accounts for complex physical interactions in a challenging toy experiment. Further, we find that modelling interactions leads to consistent performance gains in tracking as well as future trajectory prediction on three real-world datasets (MOTChallenge, UA-DETRAC, and Stanford Drone dataset), particularly in the presence of ego-motion, occlusions, crowded scenes, and faulty sensor inputs.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesMulti-Object TrackingObjectObject TrackingRelational ReasoningTrajectory PredictionSimilar Papers 제목 키워드 기반
Multiple Object Tracking with Mixture Density Networks for Trajectory Estimation
Multiple object tracking faces several challenges that may be alleviated with trajectory information. Knowing the posterior locations of an object helps disambiguating and solving situations such as occlusions, re-identi…
Multi-Object TrackingMultiple Object TrackingObjectObject TrackingSOTFormer: A Minimal Transformer for Unified Object Tracking and Trajectory Prediction
Accurate single-object tracking and short-term motion forecasting remain challenging under occlusion, scale variation, and temporal drift, which disrupt the temporal coherence required for real-time perception. We introd…
Trajectory PredictionMotion ForecastingObject DetectionObject TrackingRecurrent Autoregressive Networks for Online Multi-Object Tracking
The main challenge of online multi-object tracking is to reliably associate object trajectories with detections in each video frame based on their tracking history. In this work, we propose the Recurrent Autoregressive N…
Multi-Object TrackingObjectObject TrackingOnline Multi-Object TrackingObject Tracking Using Spatio-Temporal Future Prediction
Occlusion is a long-standing problem that causes many modern tracking methods to be erroneous. In this paper, we address the occlusion problem by exploiting the current and future possible locations of the target object …
Future predictionObjectObject TrackingPrediction+1Improving End-to-End Object Tracking Using Relational Reasoning
Relational reasoning, the ability to model interactions and relations between objects, is valuable for robust multi-object tracking and pivotal for trajectory prediction. In this paper, we propose MOHART, a class-agnosti…
Multi-Object TrackingObjectObject TrackingPrediction+2