3M3D: Multi-view, Multi-path, Multi-representation for 3D Object Detection
3D visual perception tasks based on multi-camera images are essential for autonomous driving systems. Latest work in this field performs 3D object detection by leveraging multi-view images as an input and iteratively enhancing object queries (object proposals) by cross-attending multi-view features. However, individual backbone features are not updated with multi-view features and it stays as a mere collection of the output of the single-image backbone network. Therefore we propose 3M3D: A Multi-view, Multi-path, Multi-representation for 3D Object Detection where we update both multi-view features and query features to enhance the representation of the scene in both fine panoramic view and coarse global view. Firstly, we update multi-view features by multi-view axis self-attention. It will incorporate panoramic information in the multi-view features and enhance understanding of the global scene. Secondly, we update multi-view features by self-attention of the ROI (Region of Interest) windows which encodes local finer details in the features. It will help exchange the information not only along the multi-view axis but also along the other spatial dimension. Lastly, we leverage the fact of multi-representation of queries in different domains to further boost the performance. Here we use sparse floating queries along with dense BEV (Bird's Eye View) queries, which are later post-processed to filter duplicate detections. Moreover, we show performance improvements on nuScenes benchmark dataset on top of our baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object DetectionAutonomous DrivingObjectobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
Provable Pathways: Learning Multiple Tasks over Multiple Paths
Constructing useful representations across a large number of tasks is a key requirement for sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building a shared representation across …
Generalization BoundsDPCNet: Dual Path Multi-Excitation Collaborative Network for Facial Expression Representation Learning in Videos
Current works of facial expression learning in video consume significant computational resources to learn spatial channel feature representations and temporal relationships. To mitigate this issue, we propose a Dual Path…
Dynamic Facial Expression RecognitionRepresentation LearningMultiview Canonical Correlation Analysis for Automatic Pathological Speech Detection
Recently proposed automatic pathological speech detection approaches rely on spectrogram input representations or wav2vec2 embeddings. These representations may contain pathology irrelevant uncorrelated information, such…
Dimensionality ReductionMM-Path: Multi-modal, Multi-granularity Path Representation Learning -- Extended Version
Developing effective path representations has become increasingly essential across various fields within intelligent transportation. Although pre-trained path representation learning models have shown improved performanc…
Representation LearningSVG360: Editable Multiview Vector Graphics from a Single SVG
Scalable Vector Graphics are a standard representation for editable visual design, yet they are usually authored as single view two dimensional illustrations. This limits their use in applications that require object lev…
Video Segmentation