3D Video Object Detection with Learnable Object-Centric Global Optimization
We explore long-term temporal visual correspondence-based optimization for 3D video object detection in this work. Visual correspondence refers to one-to-one mappings for pixels across multiple images. Correspondence-based optimization is the cornerstone for 3D scene reconstruction but is less studied in 3D video object detection, because moving objects violate multi-view geometry constraints and are treated as outliers during scene reconstruction. We address this issue by treating objects as first-class citizens during correspondence-based optimization. In this work, we propose BA-Det, an end-to-end optimizable object detector with object-centric temporal correspondence learning and featuremetric object bundle adjustment. Empirically, we verify the effectiveness and efficiency of BA-Det for multiple baseline 3D detectors under various setups. Our BA-Det achieves SOTA performance on the large-scale Waymo Open Dataset (WOD) with only marginal computation cost. Our code is available at https://github.com/jiaweihe1996/BA-Det.
Code (1)
Tasks
3D Scene Reconstructionglobal-optimizationObjectobject-detectionObject DetectionVideo Object DetectionSimilar Papers 제목 키워드 기반
Rethinking Image-to-Video Adaptation: An Object-centric Perspective
Image-to-video adaptation seeks to efficiently adapt image models for use in the video domain. Instead of finetuning the entire image backbone, many image-to-video adaptation paradigms use lightweight adapters for tempor…
Action RecognitionObjectObject DiscoverySemantic Segmentation+5Enhancing Next Active Object-based Egocentric Action Anticipation with Guided Attention
Short-term action anticipation (STA) in first-person videos is a challenging task that involves understanding the next active object interactions and predicting future actions. Existing action anticipation methods have p…
Action AnticipationObjectShort-term Object Interaction AnticipationThe MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain
Wearable cameras allow to collect images and videos of humans interacting with the world. While human-object interactions have been thoroughly investigated in third person vision, the problem has been understudied in ego…
Action RecognitionActive Object DetectionHuman-Object Interaction DetectionObject+3Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionaries
We present EgoACO, a deep neural architecture for video action recognition that learns to pool action-context-object descriptors from frame level features by leveraging the verb-noun structure of action labels in egocent…
Action RecognitionObjectTemporal Action LocalizationObjectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations
3D object detection has recently become popular due to many applications in robotics, augmented reality, autonomy, and image retrieval. We introduce the Objectron dataset to advance the state of the art in 3D object dete…
3D Object Detection3D Object Tracking3D Shape RepresentationImage Retrieval+6