SphereUFormer: A U-Shaped Transformer for Spherical 360 Perception

2024-12-09 · CVPR 2025 1 · Yaniv Benny, Lior Wolf

This paper proposes a novel method for omnidirectional 360$\degree$ perception. Most common previous methods relied on equirectangular projection. This representation is easily applicable to 2D operation layers but introduces distortions into the image. Other methods attempted to remove the distortions by maintaining a sphere representation but relied on complicated convolution kernels that failed to show competitive results. In this work, we introduce a transformer-based architecture that, by incorporating a novel ``Spherical Local Self-Attention'' and other spherically-oriented modules, successfully operates in the spherical domain and outperforms the state-of-the-art in 360$\degree$ perception benchmarks for depth estimation and semantic segmentation.

📄 PDF Abstract BibTeX arXiv:2412.06968

Code (0)

등록된 구현이 없습니다.

Tasks

Depth Estimation Semantic Segmentation

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

SO3UFormer: Learning Intrinsic Spherical Features for Rotation-Robust Panoramic Dense Prediction

2026-02-26 · Qinfeng Zhu, Yunxi Jiang, Lei Fan arxiv

Panoramic dense-prediction models, spanning semantic segmentation and depth estimation, are typically trained under a strict gravity-aligned assumption. Real-world captures, however, routinely violate it: handheld device…

Semantic Segmentation

SGFormer: Spherical Geometry Transformer for 360 Depth Estimation

2024-04-23 · Junsong Zhang, Zisong Chen, Chunyu Lin, Lang Nie 외

Panoramic distortion poses a significant challenge in 360 depth estimation, particularly pronounced at the north and south poles. Existing methods either adopt a bi-projection fusion strategy to remove distortions or mod…

Decoder Depth Estimation Position

Spherical Interpolated Convolutional Network with Distance-Feature Density for 3D Semantic Segmentation of Point Clouds

2020-11-27 · Guangming Wang, Yehui Yang, Huixin Zhang, Zhe Liu 외

The semantic segmentation of point clouds is an important part of the environment perception for robots. However, it is difficult to directly adopt the traditional 3D convolution kernel to extract features from raw 3D po…

3D Semantic Segmentation Feature Correlation Semantic Segmentation

SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentation

2023-06-06 · XueWei Li, Tao Wu, Zhongang Qi, Gaoang Wang 외

As an important and challenging problem in computer vision, PAnoramic Semantic Segmentation (PASS) gives complete scene perception based on an ultra-wide angle of view. Usually, prevalent PASS methods with 2D panoramic i…

Semantic Segmentation

Merging $K$-means with hierarchical clustering for identifying general-shaped groups

2017-12-23 · Anna D. Peterson, Arka P. Ghosh, Ranjan Maitra

Clustering partitions a dataset such that observations placed together in a group are similar but different from those in other groups. Hierarchical and $K$-means clustering are two approaches but have different strength…

Clustering Density Estimation Speech Synthesis