Mahalanobis Distance-based Multi-view Optimal Transport for Multi-view Crowd Localization
Multi-view crowd localization predicts the ground locations of all people in the scene. Typical methods usually estimate the crowd density maps on the ground plane first, and then obtain the crowd locations. However, the performance of existing methods is limited by the ambiguity of the density maps in crowded areas, where local peaks can be smoothed away. To mitigate the weakness of density map supervision, optimal transport-based point supervision methods have been proposed in the single-image crowd localization tasks, but have not been explored for multi-view crowd localization yet. Thus, in this paper, we propose a novel Mahalanobis distance-based multi-view optimal transport (M-MVOT) loss specifically designed for multi-view crowd localization. First, we replace the Euclidean-based transport cost with the Mahalanobis distance, which defines elliptical iso-contours in the cost function whose long-axis and short-axis directions are guided by the view ray direction. Second, the object-to-camera distance in each view is used to adjust the optimal transport cost of each location further, where the wrong predictions far away from the camera are more heavily penalized. Finally, we propose a strategy to consider all the input camera views in the model loss (M-MVOT) by computing the optimal transport cost for each ground-truth point based on its closest camera. Experiments demonstrate the advantage of the proposed method over density map-based or common Euclidean distance-based optimal transport loss on several multi-view crowd localization datasets. Project page: https://vcc.tech/research/2024/MVOT.
Code (1)
Tasks
Multiview DetectionSimilar Papers 제목 키워드 기반
Efficient Robust Optimal Transport with Application to Multi-Label Classification
Optimal transport (OT) is a powerful geometric tool for comparing two distributions and has been employed in various machine learning applications. In this work, we propose a novel OT formulation that takes feature corre…
Multi-class ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONTAGReliable Modeling of Distribution Shifts via Displacement-Reshaped Optimal Transport
Optimal transport (OT) is a central framework for modeling distribution shifts. Because OT compares distributions directly in input space, a well-designed ground metric between observations is essential to ensure that th…
Hierarchical Optimal Transport for Robust Multi-View Learning
Traditional multi-view learning methods often rely on two assumptions: ($i$) the samples in different views are well-aligned, and ($ii$) their representations in latent space obey the same distribution. Unfortunately, th…
ClusteringMULTI-VIEW LEARNINGmetricDTW: local distance metric learning in Dynamic Time Warping
We propose to learn multiple local Mahalanobis distance metrics to perform k-nearest neighbor (kNN) classification of temporal sequences. Temporal sequences are first aligned by dynamic time warping (DTW); given the alig…
Dynamic Time WarpingGeneral ClassificationMetric LearningTemporal Sequences+2On the use of Mahalanobis distance for out-of-distribution detection with neural networks for medical imaging
Implementing neural networks for clinical use in medical applications necessitates the ability for the network to detect when input data differs significantly from the training data, with the aim of preventing unreliable…
Out-of-Distribution DetectionOut of Distribution (OOD) Detection