Multi-Augmentation for Efficient Visual Representation Learning for Self-supervised Pre-training
In recent years, self-supervised learning has been studied to deal with the limitation of available labeled-dataset. Among the major components of self-supervised learning, the data augmentation pipeline is one key factor in enhancing the resulting performance. However, most researchers manually designed the augmentation pipeline, and the limited collections of transformation may cause the lack of robustness of the learned feature representation. In this work, we proposed Multi-Augmentations for Self-Supervised Representation Learning (MA-SSRL), which fully searched for various augmentation policies to build the entire pipeline to improve the robustness of the learned feature representation. MA-SSRL successfully learns the invariant feature representation and presents an efficient, effective, and adaptable data augmentation pipeline for self-supervised pre-training on different distribution and domain datasets. MA-SSRL outperforms the previous state-of-the-art methods on transfer and semi-supervised benchmarks while requiring fewer training epochs.
Code (1)
Tasks
Data AugmentationRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
IPCL: Iterative Pseudo-Supervised Contrastive Learning to Improve Self-Supervised Feature Representation
Self-supervised learning with a contrastive batch approach has become a powerful tool for representation learning in computer vision. The performance of downstream tasks is proportional to the quality of visual features …
Contrastive LearningData Augmentationimage-classificationImage Classification+5The Impact of Spatiotemporal Augmentations on Self-Supervised Audiovisual Representation Learning
Contrastive learning of auditory and visual perception has been extremely successful when investigated individually. However, there are still major questions on how we could integrate principles learned from both domains…
Contrastive LearningRepresentation LearningSelf-Supervised LearningTime to augment self-supervised visual representation learning
Biological vision systems are unparalleled in their ability to learn visual representations without supervision. In machine learning, self-supervised learning (SSL) has led to major advances in forming object representat…
Contrastive LearningObjectRepresentation LearningSelf-Supervised LearningSelf-supervised Visualisation of Medical Image Datasets
Self-supervised learning methods based on data augmentations, such as SimCLR, BYOL, or DINO, allow obtaining semantically meaningful representations of image datasets and are widely used prior to supervised fine-tuning. …
Contrastive LearningSelf-Supervised LearningEquivariant Representation Learning for Augmentation-based Self-Supervised Learning via Image Reconstruction
Augmentation-based self-supervised learning methods have shown remarkable success in self-supervised visual representation learning, excelling in learning invariant features but often neglecting equivariant ones. This li…
Image ReconstructionRepresentation LearningSelf-Supervised Learning