AAVAE: Augmentation-Augmented Variational Autoencoders
Recent methods for self-supervised learning can be grouped into two paradigms: contrastive and non-contrastive approaches. Their success can largely be attributed to data augmentation pipelines which generate multiple views of a single input that preserve the underlying semantics. In this work, we introduce augmentation-augmented variational autoencoders (AAVAE), yet another alternative to self-supervised learning, based on autoencoding. We derive AAVAE starting from the conventional variational autoencoder (VAE), by replacing the KL divergence regularization, which is agnostic to the input domain, with data augmentations that explicitly encourage the internal representations to encode domain-specific invariances and equivariances. We empirically evaluate the proposed AAVAE on image classification, similar to how recent contrastive and non-contrastive learning algorithms have been evaluated. Our experiments confirm the effectiveness of data augmentation as a replacement for KL divergence regularization. The AAVAE outperforms the VAE by 30% on CIFAR-10, 40% on STL-10 and 45% on Imagenet. On CIFAR-10 and STL-10, the results for AAVAE are largely comparable to the state-of-the-art algorithms for self-supervised learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningData Augmentationimage-classificationImage ClassificationSelf-Supervised LearningSimilar Papers 제목 키워드 기반
AASAE: Augmentation-Augmented Stochastic Autoencoders
Recent methods for self-supervised learning can be grouped into two paradigms: contrastive and non-contrastive approaches. Their success can largely be attributed to data augmentation pipelines which generate multiple vi…
Contrastive LearningData Augmentationimage-classificationImage Classification+1Category-Learning with Context-Augmented Autoencoder
Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. Biological neural networks are known to solve this problem quite well in unsupervised manner, yet u…
Data AugmentationData Augmentation with Variational Autoencoders and Manifold Sampling
We propose a new efficient way to sample from a Variational Autoencoder in the challenging low sample size setting. This method reveals particularly well suited to perform data augmentation in such a low data regime and …
Data AugmentationSynthetic Data Augmentation for Medical Audio Classification: A Preliminary Evaluation
Medical audio classification remains challenging due to low signal-to-noise ratios, subtle discriminative features, and substantial intra-class variability, often compounded by class imbalance and limited training data. …
Audio ClassificationData AugmentationImproving SMOTE via Fusing Conditional VAE for Data-adaptive Noise Filtering
Recent advances in a generative neural network model extend the development of data augmentation methods. However, the augmentation methods based on the modern generative models fail to achieve notable performance for cl…
ClassificationData Augmentationimbalanced classification