Exploring Context with Deep Structured models for Semantic Segmentation

2016-03-10 · Guosheng Lin, Chunhua Shen, Anton Van Den Hengel, Ian Reid

State-of-the-art semantic image segmentation methods are mostly based on training deep convolutional neural networks (CNNs). In this work, we proffer to improve semantic segmentation with the use of contextual information. In particular, we explore patch-patch' context and patch-background' context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets including $NYUDv2$, $PASCAL$-$VOC2012$, $Cityscapes$, $PASCAL$-$Context$, $SUN$-$RGBD$, $SIFT$-$flow$, and $KITTI$ datasets. Particularly, we report an intersection-over-union score of $77.8$ on the $PASCAL$-$VOC2012$ dataset.

📄 PDF Abstract BibTeX arXiv:1603.03183

Code (0)

등록된 구현이 없습니다.

Tasks

Image Segmentation Segmentation Semantic Segmentation

Methods 이 논문이 사용한 방법론

CRF Conditional Random Fields or CRFs are a type of probabilistic graph model that take neighboring sample context into account for tasks like classification. Prediction is…

Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds

2018-02-05 · Francis Engelmann, Theodora Kontogianni, Alexander Hermans, Bastian Leibe

Deep learning approaches have made tremendous progress in the field of semantic segmentation over the past few years. However, most current approaches operate in the 2D image space. Direct semantic segmentation of unstru…

3D Semantic Segmentation Segmentation Semantic Segmentation

CTNet: Context-based Tandem Network for Semantic Segmentation

2021-04-20 · Zechao Li, Yanpeng Sun, Jinhui Tang

Contextual information has been shown to be powerful for semantic segmentation. This work proposes a novel Context-based Tandem Network (CTNet) by interactively exploring the spatial contextual information and the channe…

Segmentation Semantic Segmentation

Exploring Cross-Image Pixel Contrast for Semantic Segmentation

2021-01-28 · ICCV 2021 10 · Wenguan Wang, Tianfei Zhou, Fisher Yu, Jifeng Dai 외

Current semantic segmentation methods focus only on mining "local" context, i.e., dependencies between pixels within individual images, by context-aggregation modules (e.g., dilated convolution, neural attention) or stru…

Metric Learning Optical Character Recognition (OCR)Representation Learning Segmentation+1

Efficient piecewise training of deep structured models for semantic segmentation

2015-04-04 · CVPR 2016 6 · Guosheng Lin, Chunhua Shen, Anton van dan Hengel, Ian Reid

Recent advances in semantic image segmentation have mostly been achieved by training deep convolutional neural networks (CNNs). We show how to improve semantic segmentation through the use of contextual information; spec…