Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagery
Can we improve detection in the thermal domain by borrowing features from rich domains like visual RGB? In this paper, we propose a pseudo-multimodal object detector trained on natural image domain data to help improve the performance of object detection in thermal images. We assume access to a large-scale dataset in the visual RGB domain and relatively smaller dataset (in terms of instances) in the thermal domain, as is common today. We propose the use of well-known image-to-image translation frameworks to generate pseudo-RGB equivalents of a given thermal image and then use a multi-modal architecture for object detection in the thermal image. We show that our framework outperforms existing benchmarks without the explicit need for paired training examples from the two domains. We also show that our framework has the ability to learn with less data from thermal domain when using our approach. Our code and pre-trained models are made available at https://github.com/tdchaitanya/MMTOD
Code (1)
Tasks
Image-to-Image TranslationObjectobject-detectionObject DetectionTranslationSimilar Papers 제목 키워드 기반
From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
3D visual grounding has made notable progress in localizing objects within complex 3D scenes. However, grounding referring expressions beyond objects in 3D scenes remains unexplored. In this paper, we introduce Anywhere3…
3D visual groundingObjectReferring ExpressionSpatial Reasoning+1PC$^2$: Pseudo-Classification Based Pseudo-Captioning for Noisy Correspondence Learning in Cross-Modal Retrieval
In the realm of cross-modal retrieval, seamlessly integrating diverse modalities within multimedia remains a formidable challenge, especially given the complexities introduced by noisy correspondence learning (NCL). Such…
Cross-modal retrieval with noisy correspondenceImage-text RetrievalAnywhereDoor: Multi-Target Backdoor Attacks on Object Detection
As object detection becomes integral to many safety-critical applications, understanding its vulnerabilities is essential. Backdoor attacks, in particular, pose a serious threat by implanting hidden triggers in victim mo…
Backdoor AttackMulti-Task LearningObject DetectionAnywhereDoor: Multi-Target Backdoor Attacks on Object Detection
As object detection becomes integral to many safety-critical applications, understanding its vulnerabilities is essential. Backdoor attacks, in particular, pose a serious threat by implanting hidden triggers in victim mo…
Backdoor AttackMulti-Task LearningObject DetectionOpen Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation
Many methods of semantic image segmentation have borrowed the success of open compound domain adaptation. They minimize the style gap between the images of source and target domains, more easily predicting the accurate p…
Domain AdaptationImage SegmentationObjectSemantic Segmentation