Auto-Vocabulary 3D Object Detection
Open-vocabulary 3D object detection methods are able to localize 3D boxes of classes unseen during training. Despite the name, existing methods rely on user-specified classes both at training and inference. We propose to study Auto-Vocabulary 3D Object Detection (AV3DOD), where the classes are automatically generated for the detected objects without any user input. To this end, we introduce Semantic Score (SS) to evaluate the quality of the generated class names. We then develop a novel framework, AV3DOD, which leverages 2D vision-language models (VLMs) to generate rich semantic candidates through image captioning, pseudo 3D box generation, and feature-space semantics expansion. AV3DOD achieves the state-of-the-art (SOTA) performance on both localization (mAP) and semantic quality (SS) on the ScanNetV2 and SUNRGB-D datasets. Notably, it surpasses the SOTA, CoDA, by 3.48 overall mAP and attains a 24.5% relative improvement in SS on ScanNetV2.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object DetectionImage CaptioningResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #200 | 3D Object Detection | ScanNetV2 | Semantic | mAP@0.25: 3.48 |
Similar Papers 제목 키워드 기반
Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data
Open-vocabulary 3D object detection has recently attracted considerable attention due to its broad applications in autonomous driving and robotics, which aims to effectively recognize novel classes in previously unseen d…
3D Object DetectionAutonomous DrivingMonocular 3D Object Detectionobject-detection+1Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community
Object detection, particularly open-vocabulary object detection, plays a crucial role in Earth sciences, such as environmental monitoring, natural disaster assessment, and land-use planning. However, existing open-vocabu…
Novel ConceptsObjectobject-detectionObject Detection+3Identifying the Unknown: Prompt-Free Open Vocabulary Anomaly Recognition for Robot-Object Interaction
Robots operating in real-world environments must in general be able to recognize previously unseen objects. As robotic systems move toward open-world autonomy, there is a growing, yet largely unmet, need for open vocabul…
Anomaly DetectionFine-Grained Open-Vocabulary Object Recognition via User-Guided Segmentation
Recent advent of vision-based foundation models has enabled efficient and high-quality object detection at ease. Despite the success of previous studies, object detection models face limitations on capturing small compon…
Objectobject-detectionObject DetectionObject Recognition+2LLM-Guided Agentic Object Detection for Open-World Understanding
Object detection traditionally relies on fixed category sets, requiring costly re-training to handle novel objects. While Open-World and Open-Vocabulary Object Detection (OWOD and OVOD) improve flexibility, OWOD lacks se…
Object Detection