Adversarial Contrastive Self-Supervised Learning
Recently, learning from vast unlabeled data, especially self-supervised learning, has been emerging and attracted widespread attention. Self-supervised learning followed by the supervised fine-tuning on a few labeled examples can significantly improve label efficiency and outperform standard supervised training using fully annotated data. In this work, we present a novel self-supervised deep learning paradigm based on online hard negative pair mining. Specifically, we design a student-teacher network to generate multi-view of the data for self-supervised learning and integrate hard negative pair mining into the training. Then we derive a new triplet-like loss considering both positive sample pairs and mined hard negative sample pairs. Extensive experiments demonstrate the effectiveness of the proposed method and its components on ILSVRC-2012.
Code (0)
등록된 구현이 없습니다.
Tasks
Self-Supervised LearningTripletSimilar Papers 제목 키워드 기반
PointACL:Adversarial Contrastive Learning for Robust Point Clouds Representation under Adversarial Attack
Despite recent success of self-supervised based contrastive learning model for 3D point clouds representation, the adversarial robustness of such pre-trained models raised concerns. Adversarial contrastive learning (ACL)…
3D ClassificationAdversarial AttackAdversarial RobustnessContrastive LearningAdversarial Self-Supervised Contrastive Learning
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions, which are then used to augment the training of the model for improved robustness. While…
Adversarial AttackContrastive LearningSelf-Supervised LearningTransfer LearningRobust Pre-Training by Adversarial Contrastive Learning
Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness In this work, we improve robustness-aware self-supervised pre-training by learni…
Adversarial RobustnessContrastive LearningEffective Targeted Attacks for Adversarial Self-Supervised Learning
Recently, unsupervised adversarial training (AT) has been highlighted as a means of achieving robustness in models without any label information. Previous studies in unsupervised AT have mostly focused on implementing se…
Adversarial AttackSelf-Supervised LearningCADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning
Handling out-of-distribution (OOD) samples has become a major stake in the real-world deployment of machine learning systems. This work explores the use of self-supervised contrastive learning to the simultaneous detecti…
Anomaly DetectionContrastive LearningOut-of-Distribution Detection