A Simple Framework for Text-Supervised Semantic Segmentation

2023-01-01 · CVPR 2023 1 · Muyang Yi, Quan Cui, Hao Wu, Cheng Yang, Osamu Yoshie, Hongtao Lu

Text-supervised semantic segmentation is a novel research topic that allows semantic segments to emerge with image-text contrasting. However, pioneering methods could be subject to specifically designed network architectures. This paper shows that a vanilla contrastive language-image pre-training (CLIP) model is an effective text-supervised semantic segmentor by itself. First, we reveal that a vanilla CLIP is inferior to localization and segmentation due to its optimization being driven by densely aligning visual and language representations. Second, we propose the locality-driven alignment (LoDA) to address the problem, where CLIP optimization is driven by sparsely aligning local representations. Third, we propose a simple segmentation (SimSeg) framework. LoDA and SimSeg jointly ameliorate a vanilla CLIP to produce impressive semantic segmentation results. Our method outperforms previous state-of-the-art methods on PASCAL VOC 2012, PASCAL Context and COCO datasets by large margins. Code and models are available at github.com/muyangyi/SimSeg.

📄 PDF Abstract BibTeX

Code (1)

muyangyi/simseg 공식 구현 pytorch

Tasks

Segmentation Semantic Segmentation

Methods 이 논문이 사용한 방법론

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…

Unsupervised Universal Image Segmentation

2023-12-28 · CVPR 2024 1 · Dantong Niu, Xudong Wang, Xinyang Han, Long Lian 외

Several unsupervised image segmentation approaches have been proposed which eliminate the need for dense manually-annotated segmentation masks; current models separately handle either semantic segmentation (e.g., STEGO) …

Image Segmentation Instance Segmentation Panoptic Segmentation Segmentation+8

A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation

2021-04-15 · ICCV 2021 10 · Jianlong Yuan, Yifan Liu, Chunhua Shen, Zhibin Wang 외

Recently, significant progress has been made on semantic segmentation. However, the success of supervised semantic segmentation typically relies on a large amount of labelled data, which is time-consuming and costly to o…

Data Augmentation image-classification Image Classification Pseudo Label+3

Learning to Rank Semantic Coherence for Topic Segmentation

2017-09-01 · EMNLP 2017 9 · Liang Wang, Sujian Li, Yajuan Lv, Houfeng Wang

Topic segmentation plays an important role for discourse parsing and information retrieval. Due to the absence of training data, previous work mainly adopts unsupervised methods to rank semantic coherence between paragra…

Common Sense Reasoning Discourse Parsing Information Retrieval Learning-To-Rank+3

SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentation

2024-06-12 · Chanda Grover Kamra, Indra Deep Mastan, Nitin Kumar, Debayan Gupta

Recent developments in self-supervised learning (SSL) have made it possible to learn data representations without the need for annotations. Inspired by the non-contrastive SSL approach (SimSiam), we introduce a novel fra…

Image Segmentation Segmentation Self-Supervised Learning Semantic Segmentation+1

Convolutional Simplex Projection Network (CSPN) for Weakly Supervised Semantic Segmentation

2018-07-24 · Rania Briq, Michael Moeller, Juergen Gall

Weakly supervised semantic segmentation has been a subject of increased interest due to the scarcity of fully annotated images. We introduce a new approach for solving weakly supervised semantic segmentation with deep Co…

Segmentation Semantic Segmentation Weakly supervised Semantic Segmentation Weakly-Supervised Semantic Segmentation