Referring Multi-Object Tracking
Existing referring understanding tasks tend to involve the detection of a single text-referred object. In this paper, we propose a new and general referring understanding task, termed referring multi-object tracking (RMOT). Its core idea is to employ a language expression as a semantic cue to guide the prediction of multi-object tracking. To the best of our knowledge, it is the first work to achieve an arbitrary number of referent object predictions in videos. To push forward RMOT, we construct one benchmark with scalable expressions based on KITTI, named Refer-KITTI. Specifically, it provides 18 videos with 818 expressions, and each expression in a video is annotated with an average of 10.7 objects. Further, we develop a transformer-based architecture TransRMOT to tackle the new task in an online manner, which achieves impressive detection performance and outperforms other counterparts. The dataset and code will be available at https://github.com/wudongming97/RMOT.
Code (1)
Tasks
Multi-Object TrackingObjectObject TrackingReferring Multi-Object TrackingSimilar Papers 제목 키워드 기반
Cross-View Referring Multi-Object Tracking
Referring Multi-Object Tracking (RMOT) is an important topic in the current tracking field. Its task form is to guide the tracker to track objects that match the language description. Current research mainly focuses on r…
Cross-view Referring Multi-Object TrackingMulti-Object TrackingObjectObject Tracking+1STORM: End-to-End Referring Multi-Object Tracking in Videos
Referring multi-object tracking (RMOT) is a task of associating all the objects in a video that semantically match with given textual queries or referring expressions. Existing RMOT approaches decompose object grounding …
Multi-Object TrackingORMOT: A Dataset and Framework for Omnidirectional Referring Multi-Object Tracking
Multi-Object Tracking (MOT) is a fundamental task in computer vision, aiming to track targets across video frames. Existing MOT methods perform well in general visual scenes, but face significant challenges and limitatio…
Multi-Object TrackingDRMOT: A Dataset and Framework for RGBD Referring Multi-Object Tracking
Referring Multi-Object Tracking (RMOT) aims to track specific targets based on language descriptions and is vital for interactive AI systems such as robotics and autonomous driving. However, existing RMOT models rely sol…
Multi-Object TrackingAutonomous DrivingCD-RMOT-Bench: Benchmarking the Cross-Domain Referring Multi-Object Tracking
Referring multi-object tracking (RMOT) extends tracking from category-driven perception to language-guided understanding by grounding object trajectories in natural-language expressions. Despite recent progress, existing…
Multi-Object TrackingObject Detection