Bipartite Conditional Random Fields for Panoptic Segmentation

2019-12-11 · Sadeep Jayasumana, Kanchana Ranasinghe, Mayuka Jayawardhana, Sahan Liyanaarachchi, Harsha Ranasinghe

We tackle the panoptic segmentation problem with a conditional random field (CRF) model. Panoptic segmentation involves assigning a semantic label and an instance label to each pixel of a given image. At each pixel, the semantic label and the instance label should be compatible. Furthermore, a good panoptic segmentation should have a number of other desirable properties such as the spatial and color consistency of the labeling (similar looking neighboring pixels should have the same semantic label and the instance label). To tackle this problem, we propose a CRF model, named Bipartite CRF or BCRF, with two types of random variables for semantic and instance labels. In this formulation, various energies are defined within and across the two types of random variables to encourage a consistent panoptic segmentation. We propose a mean-field-based efficient inference algorithm for solving the CRF and empirically show its convergence properties. This algorithm is fully differentiable, and therefore, BCRF inference can be included as a trainable module in a deep network. In the experimental evaluation, we quantitatively and qualitatively show that the BCRF yields superior panoptic segmentation results in practice.

📄 PDF Abstract BibTeX arXiv:1912.05307

Code (1)

sahan-liyanaarachchi/bcrf-detectron 공식 구현 pytorch

Tasks

Panoptic Segmentation Segmentation

Methods 이 논문이 사용한 방법론

CRF Conditional Random Fields or CRFs are a type of probabilistic graph model that take neighboring sample context into account for tasks like classification. Prediction is…

K-Net: Towards Unified Image Segmentation

2021-06-28 · NeurIPS 2021 12 · Wenwei Zhang, Jiangmiao Pang, Kai Chen, Chen Change Loy

Semantic, instance, and panoptic segmentations have been addressed using different and specialized frameworks despite their underlying connections. This paper presents a unified, simple, and effective framework for these…

Image Segmentation Instance Segmentation Panoptic Segmentation Segmentation+1

PUPS: Point Cloud Unified Panoptic Segmentation

2023-02-13 · Shihao Su, Jianyun Xu, Huanyu Wang, Zhenwei Miao 외

Point cloud panoptic segmentation is a challenging task that seeks a holistic solution for both semantic and instance segmentation to predict groupings of coherent points. Previous approaches treat semantic and instance …

Decoder Instance Segmentation Panoptic Segmentation Segmentation+1

Instance and Panoptic Segmentation Using Conditional Convolutions

2021-02-05 · Zhi Tian, BoWen Zhang, Hao Chen, Chunhua Shen

We propose a simple yet effective framework for instance and panoptic segmentation, termed CondInst (conditional convolutions for instance and panoptic segmentation). In the literature, top-performing instance segmentati…

Instance Segmentation Panoptic Segmentation Segmentation Semantic Segmentation

Panoptic Lifting for 3D Scene Understanding with Neural Fields

2022-12-19 · CVPR 2023 1 · Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Buló, Norman Müller 외

We propose Panoptic Lifting, a novel approach for learning panoptic 3D volumetric representations from images of in-the-wild scenes. Once trained, our model can render color images together with 3D-consistent panoptic se…

2D Panoptic Segmentation Panoptic Segmentation Scene Understanding Segmentation

PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things

2019-03-04 · Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji

We propose PanopticFusion, a novel online volumetric semantic mapping system at the level of stuff and things. In contrast to previous semantic mapping systems, PanopticFusion is able to densely predict class labels of a…

3D Instance Segmentation Instance Segmentation Panoptic Segmentation Semantic Segmentation