ViCE: Improving Dense Representation Learning by Superpixelization and Contrasting Cluster Assignment
Recent self-supervised models have demonstrated equal or better performance than supervised methods, opening for AI systems to learn visual representations from practically unlimited data. However, these methods are typically classification-based and thus ineffective for learning high-resolution feature maps that preserve precise spatial information. This work introduces superpixels to improve self-supervised learning of dense semantically rich visual concept embeddings. Decomposing images into a small set of visually coherent regions reduces the computational complexity by $\mathcal{O}(1000)$ while preserving detail. We experimentally show that contrasting over regions improves the effectiveness of contrastive learning methods, extends their applicability to high-resolution images, improves overclustering performance, superpixels are better than grids, and regional masking improves performance. The expressiveness of our dense embeddings is demonstrated by improving the SOTA unsupervised semantic segmentation benchmark on Cityscapes, and for convolutional models on COCO.
Code (1)
Tasks
Contrastive LearningDomain GeneralizationLearning Word EmbeddingsRepresentation LearningSelf-Supervised LearningSemantic SegmentationSuperpixelsUnsupervised Semantic SegmentationWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mining Contrasting Quasi-Clique Patterns
Mining dense quasi-cliques is a well-known clustering task with applications ranging from social networks over collaboration graphs to document analysis. Recent work has extended this task to multiple graphs; i.e. the go…
ClusteringHow does Contrastive Learning Organize Images?
Contrastive learning, a dominant self-supervised technique, emphasizes similarity in representations between augmentations of the same input and dissimilarity for different ones. Although low contrastive loss often corre…
ClusteringContrastive LearningPatch-Level Contrasting without Patch Correspondence for Accurate and Dense Contrastive Representation Learning
We propose ADCLR: A ccurate and D ense Contrastive Representation Learning, a novel self-supervised learning framework for learning accurate and dense vision representation. To extract spatial-sensitive information, ADCL…
Instance Segmentationobject-detectionObject DetectionRepresentation Learning+2Masked Clustering Prediction for Unsupervised Point Cloud Pre-training
Vision transformers (ViTs) have recently been widely applied to 3D point cloud understanding, with masked autoencoding as the predominant pre-training paradigm. However, the challenge of learning dense and informative se…
Unsupervised Pre-trainingSemantic SegmentationContrastive LearningObject DetectionOpen-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding
To bridge the gap between supervised semantic segmentation and real-world applications that acquires one model to recognize arbitrary new concepts, recent zero-shot segmentation attracts a lot of attention by exploring t…
ClusteringOnline ClusteringSegmentationSemantic Segmentation+1