Generalizing Multiple Object Tracking to Unseen Domains by Introducing Natural Language Representation
Although existing multi-object tracking (MOT) algorithms have obtained competitive performance on various benchmarks, almost all of them train and validate models on the same domain. The domain generalization problem of MOT is hardly studied. To bridge this gap, we first draw the observation that the high-level information contained in natural language is domain invariant to different tracking domains. Based on this observation, we propose to introduce natural language representation into visual MOT models for boosting the domain generalization ability. However, it is infeasible to label every tracking target with a textual description. To tackle this problem, we design two modules, namely visual context prompting (VCP) and visual-language mixing (VLM). Specifically, VCP generates visual prompts based on the input frames. VLM joints the information in the generated visual prompts and the textual prompts from a pre-defined Trackbook to obtain instance-level pseudo textual description, which is domain invariant to different tracking scenes. Through training models on MOT17 and validating them on MOT20, we observe that the pseudo textual descriptions generated by our proposed modules improve the generalization performance of query-based trackers by large margins.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain GeneralizationMulti-Object TrackingMultiple Object TrackingObject TrackingSimilar Papers 제목 키워드 기반
Slot Dependency Modeling for Zero-Shot Cross-Domain Dialogue State Tracking
Zero-shot learning for Dialogue State Tracking (DST) focuses on generalizing to an unseen domain without the expense of collecting in domain data. However, previous zero-shot DST methods ignore the slot dependencies in a…
Dialogue State TrackingZero-Shot LearningDynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking
Dialogue State Tracking (DST) aims to keep track of users' intentions during the course of a conversation. In DST, modelling the relations among domains and slots is still an under-studied problem. Existing approaches th…
DecoderDialogue State TrackingMulti-domain Dialogue State TrackingTransfer LearningDynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking
Dialogue State Tracking (DST) aims to keep track of users' intentions during the course of a conversation. In DST, modelling the relations among domains and slots is still an under-studied problem. Existing approaches th…
DecoderDialogue State TrackingMulti-domain Dialogue State TrackingTransfer LearningImproving Generalization Ability for 3D Object Detection by Learning Sparsity-invariant Features
In autonomous driving, 3D object detection is essential for accurately identifying and tracking objects. Despite the continuous development of various technologies for this task, a significant drawback is observed in mos…
3D Object DetectionAutonomous DrivingDomain Adaptationobject-detection+1Universal Cross-Domain Retrieval: Generalizing Across Classes and Domains
In this work, for the first time, we address the problem of universal cross-domain retrieval, where the test data can belong to classes or domains which are unseen during training. Due to dynamically increasing number of…
Retrieval