Representation Alignment Contrastive Regularization for Multi-Object Tracking
Achieving high-performance in multi-object tracking algorithms heavily relies on modeling spatio-temporal relationships during the data association stage. Mainstream approaches encompass rule-based and deep learning-based methods for spatio-temporal relationship modeling. While the former relies on physical motion laws, offering wider applicability but yielding suboptimal results for complex object movements, the latter, though achieving high-performance, lacks interpretability and involves complex module designs. This work aims to simplify deep learning-based spatio-temporal relationship models and introduce interpretability into features for data association. Specifically, a lightweight single-layer transformer encoder is utilized to model spatio-temporal relationships. To make features more interpretative, two contrastive regularization losses based on representation alignment are proposed, derived from spatio-temporal consistency rules. By applying weighted summation to affinity matrices, the aligned features can seamlessly integrate into the data association stage of the original tracking workflow. Experimental results showcase that our model enhances the majority of existing tracking networks' performance without excessive complexity, with minimal increase in training overhead and nearly negligible computational and storage costs.
Code (1)
Tasks
Multi-Object TrackingObjectObject TrackingSimilar Papers 제목 키워드 기반
Non-Contrastive Vision-Language Learning with Predictive Embedding Alignment
Vision-language models have transformed multimodal representation learning, yet dominant contrastive approaches like CLIP require large batch sizes, careful negative sampling, and extensive hyperparameter tuning. We intr…
Representation LearningDistractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning
Change captioning aims to succinctly describe the semantic change between a pair of similar images, while being immune to distractors (illumination and viewpoint changes). Under these distractors, unchanged objects often…
Caption Generationcross-modal alignmentRepresentation LearningMulti-Alignment Contrastive Learning for Enzyme--Reaction Retrieval
Identifying enzymes that catalyze target biochemical reactions is a key step in computational enzyme discovery and biocatalyst design. Recent representation-learning methods formulate this problem as enzyme--reaction mat…
Contrastive LearningUnderstanding and Constructing Latent Modality Structures in Multi-modal Representation Learning
Contrastive loss has been increasingly used in learning representations from multiple modalities. In the limit, the nature of the contrastive loss encourages modalities to exactly match each other in the latent space. Ye…
Few-Shot Image Classificationimage-classificationImage ClassificationImage-text Retrieval+9InfoNCE Induces Gaussian Distribution
Contrastive learning has become a cornerstone of modern representation learning, allowing training with massive unlabeled data for both task-specific and general (foundation) models. A prototypical loss in contrastive tr…
Representation LearningContrastive Learning