Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks
Recently, contrastive learning-based image translation methods have been proposed, which contrasts different spatial locations to enhance the spatial correspondence. However, the methods often ignore the diverse semantic relation within the images. To address this, here we propose a novel semantic relation consistency (SRC) regularization along with the decoupled contrastive learning, which utilize the diverse semantics by focusing on the heterogeneous semantics between the image patches of a single image. To further improve the performance, we present a hard negative mining by exploiting the semantic relation. We verified our method for three tasks: single-modal and multi-modal image translations, and GAN compression task for image translation. Experimental results confirmed the state-of-art performance of our method in all the three tasks.
Code (1)
Tasks
Contrastive LearningImage-to-Image TranslationRelationTranslationSimilar Papers 제목 키워드 기반
Patch-wise Graph Contrastive Learning for Image Translation
Recently, patch-wise contrastive learning is drawing attention for the image translation by exploring the semantic correspondence between the input and output images. To further explore the patch-wise topology for high-l…
Contrastive LearningGraph Neural NetworkSemantic correspondenceTranslationCross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Segmentation
Semi-supervised learning (SSL), which aims at leveraging a few labeled images and a large number of unlabeled images for network training, is beneficial for relieving the burden of data annotation in medical image segmen…
Contrastive LearningImage SegmentationLesion SegmentationMedical Image Segmentation+4Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation
This study introduces an efficacious approach, Masked Collaborative Contrast (MCC), to highlight semantic regions in weakly supervised semantic segmentation. MCC adroitly draws inspiration from masked image modeling and …
Contrastive LearningSemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationExploring Context with Deep Structured models for Semantic Segmentation
State-of-the-art semantic image segmentation methods are mostly based on training deep convolutional neural networks (CNNs). In this work, we proffer to improve semantic segmentation with the use of contextual informatio…
Image SegmentationSegmentationSemantic SegmentationCorrCLIP: Reconstructing Correlations in CLIP with Off-the-Shelf Foundation Models for Open-Vocabulary Semantic Segmentation
Open-vocabulary semantic segmentation aims to assign semantic labels to each pixel without relying on a predefined set of categories. Contrastive Language-Image Pre-training (CLIP) demonstrates outstanding zero-shot clas…
Open Vocabulary Semantic SegmentationOpen-Vocabulary Semantic SegmentationSegmentationSemantic Segmentation+3