Few-Shot Object Detection with Foundation Models
Few-shot object detection (FSOD) aims to detect objects with only a few training examples. Visual feature extraction and query-support similarity learning are the two critical components. Existing works are usually developed based on ImageNet pre-trained vision backbones and design sophisticated metric-learning networks for few-shot learning but still have inferior accuracy. In this work we study few-shot object detection using modern foundation models. First vision-only contrastive pre-trained DINOv2 model is used for the vision backbone which shows strong transferable performance without tuning the parameters. Second Large Language Model (LLM) is employed for contextualized few-shot learning with the input of all classes and query image proposals. Language instructions are carefully designed to prompt the LLM to classify each proposal in context. The contextual information include proposal-proposal relations proposal-class relations and class-class relations which can largely promote few-shot learning. We comprehensively evaluate the proposed model (FM-FSOD) in multiple FSOD benchmarks achieving state-of-the-arts performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningFew-Shot Object DetectionLanguage ModelingLanguage ModellingLarge Language ModelMetric LearningObjectobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
With the development of large language models, many remarkable linguistic systems like ChatGPT have thrived and achieved astonishing success on many tasks, showing the incredible power of foundation models. In the spirit…
3D Object DetectionImage SegmentationObjectobject-detection+2Enhance Then Search: An Augmentation-Search Strategy with Foundation Models for Cross-Domain Few-Shot Object Detection
Foundation models pretrained on extensive datasets, such as GroundingDINO and LAE-DINO, have performed remarkably in the cross-domain few-shot object detection (CD-FSOD) task. Through rigorous few-shot training, we found…
Cross-Domain Few-ShotCross-Domain Few-Shot Object DetectionData AugmentationDomain Generalization+5FSOD-VFM: Few-Shot Object Detection with Vision Foundation Models and Graph Diffusion
In this paper, we present FSOD-VFM: Few-Shot Object Detectors with Vision Foundation Models, a framework that leverages vision foundation models to tackle the challenge of few-shot object detection. FSOD-VFM integrates t…
Few-Shot Object DetectionRevisiting Few-Shot Object Detection with Vision-Language Models
The era of vision-language models (VLMs) trained on web-scale datasets challenges conventional formulations of "open-world" perception. In this work, we revisit the task of few-shot object detection (FSOD) in the context…
Autonomous VehiclesFew-Shot Object DetectionLanguage ModellingObject+2ZERO: Industry-ready Vision Foundation Model with Multi-modal Prompts
Foundation models have revolutionized AI, yet they struggle with zero-shot deployment in real-world industrial settings due to a lack of high-quality, domain-specific datasets. To bridge this gap, Superb AI introduces ZE…
Few-Shot Object Detection