Robustness of Unsupervised Representation Learning without Labels
Unsupervised representation learning leverages large unlabeled datasets and is competitive with supervised learning. But non-robust encoders may affect downstream task robustness. Recently, robust representation encoders have become of interest. Still, all prior work evaluates robustness using a downstream classification task. Instead, we propose a family of unsupervised robustness measures, which are model- and task-agnostic and label-free. We benchmark state-of-the-art representation encoders and show that none dominates the rest. We offer unsupervised extensions to the FGSM and PGD attacks. When used in adversarial training, they improve most unsupervised robustness measures, including certified robustness. We validate our results against a linear probe and show that, for MOCOv2, adversarial training results in 3 times higher certified accuracy, a 2-fold decrease in impersonation attack success rate and considerable improvements in certified robustness.
Code (1)
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Unsupervised Graph Poisoning Attack via Contrastive Loss Back-propagation
Graph contrastive learning is the state-of-the-art unsupervised graph representation learning framework and has shown comparable performance with supervised approaches. However, evaluating whether the graph contrastive l…
Adversarial AttackContrastive LearningGraph Representation LearningLink Prediction+2Unsupervised Adversarially-Robust Representation Learning on Graphs
Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Y…
Adversarial RobustnessCommunity DetectionGraph LearningGraph Representation Learning+3Reasoning-Based Refinement of Unsupervised Text Clusters with LLMs
Unsupervised methods are widely used to induce latent semantic structure from large text collections, yet their outputs often contain incoherent, redundant, or poorly grounded clusters that are difficult to validate with…
Representation LearningTopic ModelsUnsupervised Robust Domain Adaptation without Source Data
We study the problem of robust domain adaptation in the context of unavailable target labels and source data. The considered robustness is against adversarial perturbations. This paper aims at answering the question of f…
Domain Adaptationimage-classificationImage ClassificationUnsupervised Domain AdaptationTransferable Unsupervised Robust Representation Learning
Robustness is an important, and yet, under-explored aspect of unsupervised representation learning, which has seen a lot of recent developments. In this work, we address this gap by developing a novel framework: Unsuperv…
Data AugmentationRepresentation LearningTransfer Learning