Adversarial Examples can be Effective Data Augmentation for Unsupervised Machine Learning
Adversarial examples causing evasive predictions are widely used to evaluate and improve the robustness of machine learning models. However, current studies focus on supervised learning tasks, relying on the ground-truth data label, a targeted objective, or supervision from a trained classifier. In this paper, we propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation. Our framework exploits a mutual information neural estimator as an information-theoretic similarity measure to generate adversarial examples without supervision. We propose a new MinMax algorithm with provable convergence guarantees for efficient generation of unsupervised adversarial examples. Our framework can also be extended to supervised adversarial examples. When using unsupervised adversarial examples as a simple plug-in data augmentation tool for model retraining, significant improvements are consistently observed across different unsupervised tasks and datasets, including data reconstruction, representation learning, and contrastive learning. Our results show novel methods and considerable advantages in studying and improving unsupervised machine learning via adversarial examples.
Code (1)
Tasks
BIG-bench Machine LearningContrastive LearningData AugmentationRepresentation LearningSimilar Papers 제목 키워드 기반
Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime
Data augmentation is widely used in text classification, especially in the low-resource regime where a few examples for each class are available during training. Despite the success, generating data augmentations as hard…
ClassificationData Augmentationtext-classificationText ClassificationBetter Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
Pretrained language models (PLMs) perform poorly under adversarial attacks. To improve the adversarial robustness, adversarial data augmentation (ADA) has been widely adopted to cover more search space of adversarial att…
Adversarial RobustnessData AugmentationText Augmentationtext-classification+1Simple Contrastive Representation Adversarial Learning for NLP Tasks
Self-supervised learning approach like contrastive learning is attached great attention in natural language processing. It uses pairs of training data augmentations to build a classification task for an encoder with well…
Contrastive LearningNatural Language UnderstandingSelf-Supervised LearningSemantic Textual Similarity+2SeA: Semantic Adversarial Augmentation for Last Layer Features from Unsupervised Representation Learning
Deep features extracted from certain layers of a pre-trained deep model show superior performance over the conventional hand-crafted features. Compared with fine-tuning or linear probing that can explore diverse augmenta…
Representation LearningUnsupervised Detection of Adversarial Examples with Model Explanations
Deep Neural Networks (DNNs) have shown remarkable performance in a diverse range of machine learning applications. However, it is widely known that DNNs are vulnerable to simple adversarial perturbations, which causes th…
model