SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentation
Recent developments in self-supervised learning (SSL) have made it possible to learn data representations without the need for annotations. Inspired by the non-contrastive SSL approach (SimSiam), we introduce a novel framework SIMSAM to compute the Semantic Affinity Matrix, which is significant for unsupervised image segmentation. Given an image, SIMSAM first extracts features using pre-trained DINO-ViT, then projects the features to predict the correlations of dense features in a non-contrastive way. We show applications of the Semantic Affinity Matrix in object segmentation and semantic segmentation tasks. Our code is available at https://github.com/chandagrover/SimSAM.
Code (1)
Tasks
Image SegmentationSegmentationSelf-Supervised LearningSemantic SegmentationUnsupervised Image SegmentationSimilar Papers 제목 키워드 기반
Content-based Representations of audio using Siamese neural networks
In this paper, we focus on the problem of content-based retrieval for audio, which aims to retrieve all semantically similar audio recordings for a given audio clip query. This problem is similar to the problem of query …
RetrievalUnlabeled Disentangling of GANs with Guided Siamese Networks
Disentangling underlying generative factors of a data distribution is important for interpretability and generalizable representations. In this paper, we introduce two novel disentangling methods. Our first method, Unla…
Improving Semantic Segmentation via Dilated Affinity
Introducing explicit constraints on the structural predictions has been an effective way to improve the performance of semantic segmentation models. Existing methods are mainly based on insufficient hand-crafted rules th…
SegmentationSemantic SegmentationSiamese Image Modeling for Self-Supervised Vision Representation Learning
Self-supervised learning (SSL) has delivered superior performance on a variety of downstream vision tasks. Two main-stream SSL frameworks have been proposed, i.e., Instance Discrimination (ID) and Masked Image Modeling (…
Representation LearningSelf-Supervised LearningSemantic SegmentationSensitivitySimSAM: Zero-shot Medical Image Segmentation via Simulated Interaction
The recently released Segment Anything Model (SAM) has shown powerful zero-shot segmentation capabilities through a semi-automatic annotation setup in which the user can provide a prompt in the form of clicks or bounding…
Image SegmentationMedical Image SegmentationSemantic SegmentationZero Shot Segmentation