A data set for evaluating the performance of multi-class multi-object video tracking
One of the challenges in evaluating multi-object video detection, tracking and classification systems is having publically available data sets with which to compare different systems. However, the measures of performance for tracking and classification are different. Data sets that are suitable for evaluating tracking systems may not be appropriate for classification. Tracking video data sets typically only have ground truth track IDs, while classification video data sets only have ground truth class-label IDs. The former identifies the same object over multiple frames, while the latter identifies the type of object in individual frames. This paper describes an advancement of the ground truth meta-data for the DARPA Neovision2 Tower data set to allow both the evaluation of tracking and classification. The ground truth data sets presented in this paper contain unique object IDs across 5 different classes of object (Car, Bus, Truck, Person, Cyclist) for 24 videos of 871 image frames each. In addition to the object IDs and class labels, the ground truth data also contains the original bounding box coordinates together with new bounding boxes in instances where un-annotated objects were present. The unique IDs are maintained during occlusions between multiple objects or when objects re-enter the field of view. This will provide: a solid foundation for evaluating the performance of multi-object tracking of different types of objects, a straightforward comparison of tracking system performance using the standard Multi Object Tracking (MOT) framework, and classification performance using the Neovision2 metrics. These data have been hosted publically.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGeneral ClassificationMulti-Object TrackingObjectObject TrackingSimilar Papers 제목 키워드 기반
Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks
Algorithmic bias is of increasing concern, both to the research community, and society at large. Bias in AI is more abstract and unintuitive than traditional forms of discrimination and can be more difficult to detect an…
FairnessMulti-class ClassificationEvaluating Multi-label Classifiers with Noisy Labels
Multi-label classification (MLC) is a generalization of standard classification where multiple labels may be assigned to a given sample. In the real world, it is more common to deal with noisy datasets than clean dataset…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONM2G-Eval: Enhancing and Evaluating Multi-granularity Multilingual Code Generation
The rapid advancement of code large language models (LLMs) has sparked significant research interest in systematically evaluating their code generation capabilities, yet existing benchmarks predominantly assess models at…
Code GenerationA Practioner's Guide to Evaluating Entity Resolution Results
Entity resolution (ER) is the task of identifying records belonging to the same entity (e.g. individual, group) across one or multiple databases. Ironically, it has multiple names: deduplication and record linkage, among…
ClusteringEntity ResolutionGeneral ClassificationMulti-class ClassificationEvaluating multiple models using labeled and unlabeled data
It remains difficult to evaluate machine learning classifiers in the absence of a large, labeled dataset. While labeled data can be prohibitively expensive or impossible to obtain, unlabeled data is plentiful. Here, we i…
Molecular Property PredictionProperty Prediction