3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers

2022-11-27 · Changyong Shu, Jiajun Deng, Fisher Yu, Yifan Liu

Transformer-based methods have swept the benchmarks on 2D and 3D detection on images. Because tokenization before the attention mechanism drops the spatial information, positional encoding becomes critical for those methods. Recent works found that encodings based on samples of the 3D viewing rays can significantly improve the quality of multi-camera 3D object detection. We hypothesize that 3D point locations can provide more information than rays. Therefore, we introduce 3D point positional encoding, 3DPPE, to the 3D detection Transformer decoder. Although 3D measurements are not available at the inference time of monocular 3D object detection, 3DPPE uses predicted depth to approximate the real point positions. Our hybriddepth module combines direct and categorical depth to estimate the refined depth of each pixel. Despite the approximation, 3DPPE achieves 46.0 mAP and 51.4 NDS on the competitive nuScenes dataset, significantly outperforming encodings based on ray samples. We make the codes available at https://github.com/drilistbox/3DPPE.

📄 PDF Abstract BibTeX arXiv:2211.14710

Code (1)

drilistbox/3dppe 공식 구현 pytorch

Tasks

3D Object Detection Decoder Depth Estimation Monocular 3D Object Detection Monocular Depth Estimation object-detection Object Detection

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Position-Wise Feed-Forward Layer 설명 없음

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

3DPPE: 3D Point Positional Encoding for Transformer-based Multi-Camera 3D Object Detection

2023-01-01 · ICCV 2023 1 · Changyong Shu, Jiajun Deng, Fisher Yu, Yifan Liu

3D Object Detection Decoder Monocular 3D Object Detection object-detection+1

DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers

2026-06-30 · Shun Kenney, Teppei Suzuki arxiv

The remarkable scalability of Transformers has expanded their application to 3D computer vision, where camera-aware positional encoding is crucial for providing spatial cues in multi-view geometry. Recent advancements ha…

Novel View Synthesis

Cameras as Relative Positional Encoding

2025-07-14 · RuiLong Li, Brent Yi, Junchen Liu, Hang Gao 외

Transformers are increasingly prevalent for multi-view computer vision tasks, where geometric relationships between viewpoints are critical for 3D perception. To leverage these relationships, multi-view transformers must…

Depth Estimation Novel View Synthesis Stereo Depth Estimation

CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation

2026-05-13 · Seonghyun Jin, Youngmin Kim, Sunwoo Park, Jong Chul Ye arxiv

Camera-conditioned video generation requires positional encoding that remains reliable under changes in camera motion, lens configuration, and scene structure. However, existing attention-level camera encodings either pr…

Video Generation

DPPE: Dense Pose Estimation in a Plenoxels Environment using Gradient Approximation

2024-03-16 · Christopher Kolios, Yeganeh Bahoo, Sajad Saeedi

We present DPPE, a dense pose estimation algorithm that functions over a Plenoxels environment. Recent advances in neural radiance field techniques have shown that it is a powerful tool for environment representation. Mo…

Neural Rendering Pose Estimation Template Matching