PAWS-VMK: A Unified Approach To Semi-Supervised Learning And Out-of-Distribution Detection
This paper describes PAWS-VMK, a prototypical deep learning approach that obtains state-of-the-art results for image classification tasks in both a semi-supervised learning (SSL) and out-of-distribution (OOD) detection context. We consider developments in the fields of SSL, OOD detection, and computer vision foundation models to introduce a number of innovations that connect the key ideas within these works to create PAWS-VMK. These innovations include (1) parametric von Mises-Fisher Stochastic Neighbour Embedding (vMF-SNE) to initialise a projection head for SSL using the high-quality embeddings of the foundation model; (2) the PAWS-MixMatch loss, that creates more compact embeddings and obtains higher accuracy in comparison to the consistency loss used in PAWS and (3) simple $k$-Means prototype selection (SKMPS), a simple technique that obtains competitive performance with more complex unsupervised label selection approaches. PAWS-VMK sets new benchmarks in semi-supervised learning for CIFAR-10 (99.2%) and CIFAR-100 (89.8%) with four labelled instances per class, and Food-101 (90.1%) with two labelled instances per class. We also observe that PAWS-VMK can efficiently detect OOD samples in a manner that is competitive with specialised methods specifically designed for this purpose, achieving 93.1/98.0 and 95.2/96.3 on the CIFAR-10 and CIFAR-100 OpenOOD benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Benchmarkingimage-classificationImage ClassificationOut-of-Distribution DetectionOut of Distribution (OOD) DetectionPrototype SelectionSimilar Papers 제목 키워드 기반
RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data
Semi-supervised learning aims to train a model using limited labels. State-of-the-art semi-supervised methods for image classification such as PAWS rely on self-supervised representations learned with large-scale unlabel…
Density Estimationimage-classificationImage ClassificationRepresentation LearningSemi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assignments with support samples (PAWS). The method trains a model to minimize a consistency loss, which ensures that different views of the same unlabele…
Image ClassificationPseudo LabelSemi-Supervised Image ClassificationRuPAWS: A Russian Adversarial Dataset for Paraphrase Identification
Paraphrase identification task can be easily challenged by changing word order, e.g. as in “Can a good person become bad?”. While for English this problem was tackled by the PAWS dataset (Zhang et al., 2019), datasets fo…
Paraphrase IdentificationCold PAWS: Unsupervised class discovery and addressing the cold-start problem for semi-supervised learning
In many machine learning applications, labeling datasets can be an arduous and time-consuming task. Although research has shown that semi-supervised learning techniques can achieve high accuracy with very few labels with…
Self-Supervised LearningConstrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning
We are interested in representation learning in self-supervised, supervised, and semi-supervised settings. Some recent self-supervised learning methods like mean-shift (MSF) cluster images by pulling the embedding of a q…
Pseudo LabelRepresentation LearningSelf-Supervised Learning