Indoor Scene Parsing With Instance Segmentation, Semantic Labeling and Support Relationship Inference

2017-07-01 · CVPR 2017 7 · Wei Zhuo, Mathieu Salzmann, Xuming He, Miaomiao Liu

Over the years, indoor scene parsing has attracted a growing interest in the computer vision community. Existing methods have typically focused on diverse subtasks of this challenging problem. In particular, while some of them aim at segmenting the image into regions, such as object or surface instances, others aim at inferring the semantic labels of given regions, or their support relationships. These different tasks are typically treated as separate ones. However, they bear strong connections: good regions should respect the semantic labels; support can only be defined for meaningful regions; support relationships strongly depend on semantics. In this paper, we therefore introduce an approach to jointly segment the instances and infer their semantic labels and support relationships from a single input image. By exploiting a hierarchical segmentation, we formulate our problem as that of jointly finding the regions in the hierarchy that correspond to instances and estimating their class labels and pairwise support relationships. We express this via a Markov Random Field, which allows us to further encode links between the different types of variables. Inference in this model can be done exactly via integer linear programming, and we learn its parameters in a structural SVM framework. Our experiments on NYUv2 demonstrate the benefits of reasoning jointly about all these subtasks of indoor scene parsing.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Instance Segmentation Scene Parsing Semantic Segmentation

Towards Panoptic 3D Parsing for Single Image in the Wild

2021-11-04 · Sainan Liu, Vincent Nguyen, Yuan Gao, Subarna Tripathi 외

Performing single image holistic understanding and 3D reconstruction is a central task in computer vision. This paper presents an integrated system that performs dense scene labeling, object detection, instance segmentat…

3D Reconstruction 3D Shape Reconstruction Autonomous Driving Depth Estimation+7

Unified Semantic Transformer for 3D Scene Understanding

2025-12-16 · Sebastian Koch, Johanna Wald, Hidenobu Matsuki, Pedro Hermosilla 외 arxiv

Holistic 3D scene understanding involves capturing and parsing unstructured 3D environments. Due to the inherent complexity of the real world, existing models have predominantly been developed and limited to be task-spec…

Scene Understanding Scene Segmentation

ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes

2026-01-16 · Emily Steiner, Jianhao Zheng, Henry Howard-Jenkins, Chris Xie 외 arxiv

Indoor environments evolve as objects move, appear, or leave the scene. Capturing these dynamics requires maintaining temporally consistent instance identities across intermittently captured 3D scans, even when changes a…

Instance Segmentation

Efficient Multi-Task RGB-D Scene Analysis for Indoor Environments

2022-07-10 · Daniel Seichter, Söhnke Benedikt Fischedick, Mona Köhler, Horst-Michael Groß

Semantic scene understanding is essential for mobile agents acting in various environments. Although semantic segmentation already provides a lot of information, details about individual objects as well as the general sc…

Instance Segmentation Panoptic Segmentation Scene Classification Scene Classification (unified classes)+3

IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things

2019-06-15 · ICCV 2019 10 · Cheng-Yang Fu, Tamara L. Berg, Alexander C. Berg

In this work, we present a new operator, called Instance Mask Projection (IMP), which projects a predicted Instance Segmentation as a new feature for semantic segmentation. It also supports back propagation so is trainab…

Instance Segmentation Scene Segmentation Segmentation Semantic Segmentation+1