Optimizing transformations for contrastive learning in a differentiable framework
Current contrastive learning methods use random transformations sampled from a large list of transformations, with fixed hyperparameters, to learn invariance from an unannotated database. Following previous works that introduce a small amount of supervision, we propose a framework to find optimal transformations for contrastive learning using a differentiable transformation network. Our method increases performances at low annotated data regime both in supervision accuracy and in convergence speed. In contrast to previous work, no generative model is needed for transformation optimization. Transformed images keep relevant information to solve the supervised task, here classification. Experiments were performed on 34000 2D slices of brain Magnetic Resonance Images and 11200 chest X-ray images. On both datasets, with 10% of labeled data, our model achieves better performances than a fully supervised model with 100% labels.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Transformation Invariance in Contrastive Representation Learning
We propose methods to strengthen the invariance properties of representations obtained by contrastive learning. While existing approaches implicitly induce a degree of invariance as representations are learned, we look t…
Contrastive LearningRepresentation LearningCoViews: Adaptive Augmentation Using Cooperative Views for Enhanced Contrastive Learning
Data augmentation plays a critical role in generating high-quality positive and negative pairs necessary for effective contrastive learning. However, common practices involve using a single augmentation policy repeatedly…
Contrastive LearningData AugmentationContrastive Learning for Regression on Hyperspectral Data
Contrastive learning has demonstrated great effectiveness in representation learning especially for image classification tasks. However, there is still a shortage in the studies targeting regression tasks, and more speci…
Contrastive Learningimage-classificationImage Classificationregression+1ParamCrop: Parametric Cubic Cropping for Video Contrastive Learning
The central idea of contrastive learning is to discriminate between different instances and force different views from the same instance to share the same representation. To avoid trivial solutions, augmentation plays an…
Contrastive LearningTowards Robust Graph Contrastive Learning
We study the problem of adversarially robust self-supervised learning on graphs. In the contrastive learning framework, we introduce a new method that increases the adversarial robustness of the learned representations t…
Adversarial RobustnessContrastive LearningSelf-Supervised Learning