EMPIRICAL UPPER BOUND IN OBJECT DETECTION
Object detection remains one of the most notorious open problems in computer vision. Despite large strides in accuracy and speed in recent years, modern object detectors have started to saturate on popular benchmarks. How far can we push the detection accuracy with the current deep learning tools and tricks? In this work, by employing two popular state-of-the-art object detection benchmarks, MMDetection and Detectron2, and analyzing more than 15 models over 4 large-scale datasets, we systematically determine the upper bound in AP, which is 91.6% on PASCAL VOC (test2007), 78.2% on MS COCO (val2017), and 58.9% on OpenImages (V4 validation set), regardless of the IOU. These numbers are much higher than the mAP of the best model (e.g., 58% on MS COCO according to the most recent results). Interestingly, the gap seems to be almost closed at IOU=0.5. We also analyze the role of context in object recognition and detection and find that the canonical object size leads to the best recognition accuracy. Finally, we carefully characterize the sources of errors in deep object detectors and find that classification error (confusion with other classes and misses) explains the largest fraction of errors and weighs more than localization error. Further, models frequently miss small objects, more often than medium and large ones. Our work taps into the tight relationship between object recognition and detection and offers insights to build better object detectors. Similar analyses can also be conducted for other tasks in computer vision such as for instance segmentation and object tracking. The code is available at [TBA].
Code (0)
등록된 구현이 없습니다.
Tasks
Instance SegmentationObjectobject-detectionObject DetectionObject RecognitionObject TrackingSemantic SegmentationSimilar Papers 제목 키워드 기반
Empirical Upper Bound, Error Diagnosis and Invariance Analysis of Modern Object Detectors
Object detection remains as one of the most notorious open problems in computer vision. Despite large strides in accuracy in recent years, modern object detectors have started to saturate on popular benchmarks raising th…
Objectobject-detectionObject DetectionObject RecognitionEmpirical Upper Bound in Object Detection and More
Object detection remains as one of the most notorious open problems in computer vision. Despite large strides in accuracy in recent years, modern object detectors have started to saturate on popular benchmarks raising th…
Objectobject-detectionObject DetectionOptimal Classification-based Anomaly Detection with Neural Networks: Theory and Practice
Anomaly detection is an important problem in many application areas, such as network security. Many deep learning methods for unsupervised anomaly detection produce good empirical performance but lack theoretical guarant…
Anomaly DetectionBinary ClassificationUnsupervised Anomaly DetectionChallenges in Computing and Optimizing Upper Bounds of Marginal Likelihood based on Chi-Square Divergences
Variational inference based on chi-square divergence minimization (CHIVI) provides a way to approximate a model's posterior while obtaining an upper bound on the marginal likelihood. However, in practice CHIVI relies on …
DiagnosticVariational InferenceImproving Certified Robustness via Adversarial Distillation
Certified training aims to produce models whose predictions can be formally verified against adversarial perturbations, typically by optimising upper bounds on the worst-case loss over an allowed perturbation set. For ne…