A time-weighted metric for sets of trajectories to assess multi-object tracking algorithms
This paper proposes a metric for sets of trajectories to evaluate multi-object tracking algorithms that includes time-weighted costs for localisation errors of properly detected targets, for false targets, missed targets and track switches. The proposed metric extends the metric in [1] by including weights to the costs associated to different time steps. The time-weighted costs increase the flexibility of the metric [1] to fit more applications and user preferences. We first introduce a metric based on multi-dimensional assignments, and then its linear programming relaxation, which is computable in polynomial time and is also a metric. The metrics can also be extended to metrics on random finite sets of trajectories to evaluate and rank algorithms across different scenarios, each with a ground truth set of trajectories.
Code (2)
Tasks
Multi-Object TrackingObject TrackingSimilar Papers 제목 키워드 기반
A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms
In this paper, we propose a metric on the space of finite sets of trajectories for assessing multi-target tracking algorithms in a mathematically sound way. The main use of the metric is to compare estimates of trajector…
WebGraphEval: Multi-Turn Trajectory Evaluation for Web Agents using Graph Representation
Current evaluation of web agents largely reduces to binary success metrics or conformity to a single reference trajectory, ignoring the structural diversity present in benchmark datasets. We present WebGraphEval, a frame…
Weight Prediction for Variants of Weighted Directed Networks
A weighted directed network (WDN) is a directed graph in which each edge is associated to a unique value called weight. These networks are very suitable for modeling real-world social networks in which there is an assess…
Graph EmbeddingPredictionEMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environments
Developing autonomous home robots controlled by natural language has long been a pursuit of humanity. While advancements in large language models (LLMs) and embodied intelligence make this goal closer, several challenges…
Beyond ECE: Calibrated Size Ratio, Risk Assessment, and Confidence-Weighted Metrics
Confidence calibration has been dominated by the Expected Calibration Error (ECE), a linear metric that counts calibration offset equally regardless of the confidence level at which it occurs. We show that ECE can remain…