Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds

2020-03-11 · Guangnan Wu, Zhiyi Pan, Peng Jiang, Changhe Tu

Instance segmentation in point clouds is one of the most fine-grained ways to understand the 3D scene. Due to its close relationship to semantic segmentation, many works approach these two tasks simultaneously and leverage the benefits of multi-task learning. However, most of them only considered simple strategies such as element-wise feature fusion, which may not lead to mutual promotion. In this work, we build a Bi-Directional Attention module on backbone neural networks for 3D point cloud perception, which uses similarity matrix measured from features for one task to help aggregate non-local information for the other task, avoiding the potential feature exclusion and task conflict. From comprehensive experiments and ablation studies on the S3DIS dataset and the PartNet dataset, the superiority of our method is verified. Moreover, the mechanism of how bi-directional attention module helps joint instance and semantic segmentation is also analyzed.

📄 PDF Abstract BibTeX arXiv:2003.05420

Code (1)

pumpkinnan/BAN tf

Tasks

Instance Segmentation Multi-Task Learning Segmentation Semantic Segmentation

SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering

2021-08-31 · Enxu Li, Ryan Razani, YiXuan Xu, Liu Bingbing

Panoptic segmentation aims to address semantic and instance segmentation simultaneously in a unified framework. However, an efficient solution of panoptic segmentation in applications like autonomous driving is still an …

Autonomous Driving Clustering Instance Segmentation Panoptic Segmentation+4

JSMNet Improving Indoor Point Cloud Semantic and Instance Segmentation through Self-Attention and Multiscale

2023-09-14 · Shuochen Xu, Zhenxin Zhang

The semantic understanding of indoor 3D point cloud data is crucial for a range of subsequent applications, including indoor service robots, navigation systems, and digital twin engineering. Global features are crucial f…

Instance Segmentation Segmentation Semantic Segmentation

BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation

2020-03-31 · CVPR 2020 6 · Yifeng Chen, Guangchen Lin, Songyuan Li, Bourahla Omar 외

Panoptic segmentation aims to perform instance segmentation for foreground instances and semantic segmentation for background stuff simultaneously. The typical top-down pipeline concentrates on two key issues: 1) how to …

Instance Segmentation Occlusion Handling Panoptic Segmentation Segmentation+1

Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter

2020-01-21 · Kentaro Wada, Kei Okada, Masayuki Inaba

We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into…

Instance Segmentation Segmentation Semantic Segmentation

Radar-Camera BEV Multi-Task Learning with Cross-Task Attention Bridge for Joint 3D Detection and Segmentation

2026-04-14 · Ahmet İnanç, Özgür Erkent arxiv

Bird's-eye-view (BEV) representations are the dominant paradigm for 3D perception in autonomous driving, providing a unified spatial canvas where detection and segmentation features are geometrically registered to the sa…

Multi-Task Learning Autonomous Driving