Unifying Voxel-based Representation with Transformer for 3D Object Detection

2022-06-01 · Yanwei Li, Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, Jiaya Jia

In this work, we present a unified framework for multi-modality 3D object detection, named UVTR. The proposed method aims to unify multi-modality representations in the voxel space for accurate and robust single- or cross-modality 3D detection. To this end, the modality-specific space is first designed to represent different inputs in the voxel feature space. Different from previous work, our approach preserves the voxel space without height compression to alleviate semantic ambiguity and enable spatial connections. To make full use of the inputs from different sensors, the cross-modality interaction is then proposed, including knowledge transfer and modality fusion. In this way, geometry-aware expressions in point clouds and context-rich features in images are well utilized for better performance and robustness. The transformer decoder is applied to efficiently sample features from the unified space with learnable positions, which facilitates object-level interactions. In general, UVTR presents an early attempt to represent different modalities in a unified framework. It surpasses previous work in single- or multi-modality entries. The proposed method achieves leading performance in the nuScenes test set for both object detection and the following object tracking task. Code is made publicly available at https://github.com/dvlab-research/UVTR.

📄 PDF Abstract BibTeX arXiv:2206.00630

Code (1)

dvlab-research/uvtr 공식 구현 pytorch

Tasks

3D Object Detection DecoderObjectobject-detection Object Detection Object Tracking Transfer Learning

PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer

2023-05-11 · CVPR 2023 1 · Honghui Yang, Wenxiao Wang, Minghao Chen, Binbin Lin 외

Recent Transformer-based 3D object detectors learn point cloud features either from point- or voxel-based representations. However, the former requires time-consuming sampling while the latter introduces quantization err…

Autonomous Driving Quantization

LiDARFormer: A Unified Transformer-based Multi-task Network for LiDAR Perception

2023-03-21 · Zixiang Zhou, Dongqiangzi Ye, Weijia Chen, Yufei Xie 외

There is a recent trend in the LiDAR perception field towards unifying multiple tasks in a single strong network with improved performance, as opposed to using separate networks for each task. In this paper, we introduce…

Decoder Multi-Task Learning Segmentation Semantic Segmentation

PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection

2024-05-05 · Zhaoqi Leng, Pei Sun, Tong He, Dragomir Anguelov 외

3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. In this paper, we identify that the common PointNet design introduces an information b…

3D Object Detection Object object-detection Object Detection

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds

2022-03-19 · CVPR 2022 1 · Chenhang He, Ruihuang Li, Shuai Li, Lei Zhang

Transformer has demonstrated promising performance in many 2D vision tasks. However, it is cumbersome to compute the self-attention on large-scale point cloud data because point cloud is a long sequence and unevenly dist…

3D Object Detection object-detection Object Detection

Voxel Transformer for 3D Object Detection

2021-09-06 · ICCV 2021 10 · Jiageng Mao, Yujing Xue, Minzhe Niu, Haoyue Bai 외

We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds. Conventional 3D convolutional backbones in voxel-based 3D detectors cannot efficientl…

3D Object Detection Computational Efficiency Object object-detection+2