Spatial Semantic Regularisation for Large Scale Object Detection
Large scale object detection with thousands of classes introduces the problem of many contradicting false positive detections, which have to be suppressed. Class-independent non-maximum suppression has traditionally been used for this step, but it does not scale well as the number of classes grows. Traditional non-maximum suppression does not consider label- and instance-level relationships nor does it allow an exploitation of the spatial layout of detection proposals. We propose a new multi-class spatial semantic regularisation method based on affinity propagation clustering, which simultaneously optimises across all categories and all proposed locations in the image, to improve both the localisation and categorisation of selected detection proposals. Constraints are shared across the labels through the semantic WordNet hierarchy. Our approach proves to be especially useful in large scale settings with thousands of classes, where spatial and semantic interactions are very frequent and only weakly supervised detectors can be built due to a lack of bounding box annotations. Detection experiments are conducted on the ImageNet and COCO dataset, and in settings with thousands of detected categories. Our method provides a significant precision improvement by reducing false positives, while simultaneously improving the recall.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringObjectobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
Single-shot Star-convex Polygon-based Instance Segmentation for Spatially-correlated Biomedical Objects
Biomedical images often contain objects known to be spatially correlated or nested due to their inherent properties, leading to semantic relations. Examples include cell nuclei being nested within eukaryotic cells and co…
Instance SegmentationSemantic SegmentationFast unsupervised Bayesian image segmentation with adaptive spatial regularisation
This paper presents a new Bayesian estimation technique for hidden Potts-Markov random fields with unknown regularisation parameters, with application to fast unsupervised K-class image segmentation. The technique is der…
ClusteringDenoisingImage SegmentationSegmentation+1Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models
Vision-language models (VLMs) extend large language models (LLMs) with vision encoders, enabling text generation conditioned on both images and text. However, this multimodal integration expands the attack surface by exp…
Text GenerationSceneBind: Binding What and Where Across Vision, Audio and Language
We present SceneBind, an omni-modal representation of realistic scenes with joint semantic and 3D spatial understanding across vision, audio and language. Existing omni-modal encoders excel at instance-level semantics (i…
Visual LocalizationA spectral regularisation framework for latent variable models designed for single channel applications
Latent variable models (LVMs) are commonly used to capture the underlying dependencies, patterns, and hidden structure in observed data. Source duplication is a by-product of the data hankelisation pre-processing step co…
parameter estimationTime Series