Patch-wise Graph Contrastive Learning for Image Translation
Recently, patch-wise contrastive learning is drawing attention for the image translation by exploring the semantic correspondence between the input and output images. To further explore the patch-wise topology for high-level semantic understanding, here we exploit the graph neural network to capture the topology-aware features. Specifically, we construct the graph based on the patch-wise similarity from a pretrained encoder, whose adjacency matrix is shared to enhance the consistency of patch-wise relation between the input and the output. Then, we obtain the node feature from the graph neural network, and enhance the correspondence between the nodes by increasing mutual information using the contrastive loss. In order to capture the hierarchical semantic structure, we further propose the graph pooling. Experimental results demonstrate the state-of-art results for the image translation thanks to the semantic encoding by the constructed graphs.
Code (1)
Tasks
Contrastive LearningGraph Neural NetworkSemantic correspondenceTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks
Recently, contrastive learning-based image translation methods have been proposed, which contrasts different spatial locations to enhance the spatial correspondence. However, the methods often ignore the diverse semantic…
Contrastive LearningImage-to-Image TranslationRelationTranslationSpectral Normalization and Dual Contrastive Regularization for Image-to-Image Translation
Existing image-to-image (I2I) translation methods achieve state-of-the-art performance by incorporating the patch-wise contrastive learning into Generative Adversarial Networks. However, patch-wise contrastive learning o…
Contrastive LearningImage-to-Image TranslationTranslationInstance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation
Contrastive learning shows great potential in unpaired image-to-image translation, but sometimes the translated results are in poor quality and the contents are not preserved consistently. In this paper, we uncover that …
Contrastive LearningImage-to-Image TranslationTranslationContrastive Learning Guided Latent Diffusion Model for Image-to-Image Translation
The diffusion model has demonstrated superior performance in synthesizing diverse and high-quality images for text-guided image translation. However, there remains room for improvement in both the formulation of text pro…
Contrastive LearningImage-to-Image TranslationTranslationContrastive Learning for Unpaired Image-to-Image Translation
In image-to-image translation, each patch in the output should reflect the content of the corresponding patch in the input, independent of domain. We propose a straightforward method for doing so -- maximizing mutual inf…
Contrastive LearningImage GenerationImage-to-Image TranslationTranslation