Unsupervised Domain Adaptation for Video Semantic Segmentation

2021-07-23 · Inkyu Shin, KwanYong Park, Sanghyun Woo, In So Kweon

Unsupervised Domain Adaptation for semantic segmentation has gained immense popularity since it can transfer knowledge from simulation to real (Sim2Real) by largely cutting out the laborious per pixel labeling efforts at real. In this work, we present a new video extension of this task, namely Unsupervised Domain Adaptation for Video Semantic Segmentation. As it became easy to obtain large-scale video labels through simulation, we believe attempting to maximize Sim2Real knowledge transferability is one of the promising directions for resolving the fundamental data-hungry issue in the video. To tackle this new problem, we present a novel two-phase adaptation scheme. In the first step, we exhaustively distill source domain knowledge using supervised loss functions. Simultaneously, video adversarial training (VAT) is employed to align the features from source to target utilizing video context. In the second step, we apply video self-training (VST), focusing only on the target data. To construct robust pseudo labels, we exploit the temporal information in the video, which has been rarely explored in the previous image-based self-training approaches. We set strong baseline scores on 'VIPER to CityscapeVPS' adaptation scenario. We show that our proposals significantly outperform previous image-based UDA methods both on image-level (mIoU) and video-level (VPQ) evaluation metrics.

📄 PDF Abstract BibTeX arXiv:2107.11052

Code (0)

등록된 구현이 없습니다.

Tasks

Domain Adaptation Semantic Segmentation Unsupervised Domain Adaptation Video Semantic Segmentation

Geometric Unsupervised Domain Adaptation for Semantic Segmentation

2021-03-30 · ICCV 2021 10 · Vitor Guizilini, Jie Li, Rares Ambrus, Adrien Gaidon

Simulators can efficiently generate large amounts of labeled synthetic data with perfect supervision for hard-to-label tasks like semantic segmentation. However, they introduce a domain gap that severely hurts real-world…

Depth Estimation Depth Prediction Domain Adaptation Monocular Depth Estimation+3

MoDA: Leveraging Motion Priors from Videos for Advancing Unsupervised Domain Adaptation in Semantic Segmentation

2023-09-21 · Fei Pan, Xu Yin, Seokju Lee, Axi Niu 외

Unsupervised domain adaptation (UDA) has been a potent technique to handle the lack of annotations in the target domain, particularly in semantic segmentation task. This study introduces a different UDA scenarios where t…

Domain Adaptation Image Segmentation Object Object Discovery+7

Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation

2023-03-25 · CVPR 2023 1 · Shao-Yuan Lo, Poojan Oza, Sumanth Chennupati, Alejandro Galindo 외

Unsupervised Domain Adaptation (UDA) of semantic segmentation transfers labeled source knowledge to an unlabeled target domain by relying on accessing both the source and target data. However, the access to source data i…

Contrastive Learning Domain Adaptation Semantic Segmentation Source-Free Domain Adaptation+2

Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation

2023-11-30 · Linzi Xing, Quan Tran, Fabian Caba, Franck Dernoncourt 외

Video topic segmentation unveils the coarse-grained semantic structure underlying videos and is essential for other video understanding tasks. Given the recent surge in multi-modal, relying solely on a single modality is…

Contrastive Learning Domain Adaptation Segmentation Unsupervised Domain Adaptation+1

Fully Convolutional Adaptation Networks for Semantic Segmentation

2018-04-23 · CVPR 2018 6 · Yiheng Zhang, Zhaofan Qiu, Ting Yao, Dong Liu 외

The recent advances in deep neural networks have convincingly demonstrated high capability in learning vision models on large datasets. Nevertheless, collecting expert labeled datasets especially with pixel-level annotat…

Domain Adaptation Semantic Segmentation