SAMScore: A Content Structural Similarity Metric for Image Translation Evaluation
Image translation has wide applications, such as style transfer and modality conversion, usually aiming to generate images having both high degrees of realism and faithfulness. These problems remain difficult, especially when it is important to preserve content structures. Traditional image-level similarity metrics are of limited use, since the content structures of an image are high-level, and not strongly governed by pixel-wise faithfulness to an original image. To fill this gap, we introduce SAMScore, a generic content structural similarity metric for evaluating the faithfulness of image translation models. SAMScore is based on the recent high-performance Segment Anything Model (SAM), which allows content similarity comparisons with standout accuracy. We applied SAMScore on 19 image translation tasks, and found that it is able to outperform all other competitive metrics on all tasks. We envision that SAMScore will prove to be a valuable tool that will help to drive the vibrant field of image translation, by allowing for more precise evaluations of new and evolving translation models. The code is available at https://github.com/Kent0n-Li/SAMScore.
Code (1)
Tasks
Semantic SimilaritySemantic Textual SimilarityStyle TransferTranslationSimilar Papers 제목 키워드 기반
Revisiting Self-Similarity: Structural Embedding for Image Retrieval
Despite advances in global image representation, existing image retrieval approaches rarely consider geometric structure during the global retrieval stage. In this work, we revisit the conventional self-similarity de…
Image RetrievalRetrievalEmCom-Diffusion: Probing Visual Reflection in Emergent Languages via Image Generation
Measuring the extent to which emergent languages encode the visual content of their inputs is an open problem. We refer to this property as visual reflection: the extent to which emergent messages preserve information ab…
Image GenerationCatSIM: A Categorical Image Similarity Metric
We introduce CatSIM, a new similarity metric for binary and multinary two- and three-dimensional images and volumes. CatSIM uses a structural similarity image quality paradigm and is robust to small perturbations in loca…
Image Quality AssessmentRobust Registration of Multimodal Remote Sensing Images Based on Structural Similarity
Automatic registration of multimodal remote sensing data (e.g., optical, LiDAR, SAR) is a challenging task due to the significant non-linear radiometric differences between these data. To address this problem, this paper…
Image RegistrationTemplate MatchingMRI-GAN: A Generalized Approach to Detect DeepFakes using Perceptual Image Assessment
DeepFakes are synthetic videos generated by swapping a face of an original image with the face of somebody else. In this paper, we describe our work to develop general, deep learning-based models to classify DeepFake con…
DeepFake DetectionFace SwappingGenerative Adversarial NetworkSSIM