Self-Training: A Survey
Semi-supervised algorithms aim to learn prediction functions from a small set of labeled observations and a large set of unlabeled observations. Because this framework is relevant in many applications, they have received a lot of interest in both academia and industry. Among the existing techniques, self-training methods have undoubtedly attracted greater attention in recent years. These models are designed to find the decision boundary on low density regions without making additional assumptions about the data distribution, and use the unsigned output score of a learned classifier, or its margin, as an indicator of confidence. The working principle of self-training algorithms is to learn a classifier iteratively by assigning pseudo-labels to the set of unlabeled training samples with a margin greater than a certain threshold. The pseudo-labeled examples are then used to enrich the labeled training data and to train a new classifier in conjunction with the labeled training set. In this paper, we present self-training methods for binary and multi-class classification; as well as their variants and two related approaches, namely consistency-based approaches and transductive learning. We examine the impact of significant self-training features on various methods, using different general and image classification benchmarks, and we discuss our ideas for future research in self-training. To the best of our knowledge, this is the first thorough and complete survey on this subject.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage ClassificationMulti-class ClassificationSurveyTransductive LearningSimilar Papers 제목 키워드 기반
A Survey of Self-Supervised and Few-Shot Object Detection
Labeling data is often expensive and time-consuming, especially for tasks such as object detection and instance segmentation, which require dense labeling of the image. While few-shot object detection is about training a…
Few-Shot Object DetectionInstance SegmentationObjectobject-detection+3A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks
Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, practitioners can harness unlabeled or re…
Self-Supervised LearningTransfer LearningA Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective
Graph self-supervised learning (SSL) is now a go-to method for pre-training graph foundation models (GFMs). There is a wide variety of knowledge patterns embedded in the graph data, such as node properties and clusters, …
Language ModellingLarge Language ModelSelf-Supervised LearningAMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing
Transformer-based pretrained language models (T-PTLMs) have achieved great success in almost every NLP task. The evolution of these models started with GPT and BERT. These models are built on the top of transformers, sel…
Self-Supervised LearningSurveyTransfer LearningAMMU : A Survey of Transformer-based Biomedical Pretrained Language Models
Transformer-based pretrained language models (PLMs) have started a new era in modern natural language processing (NLP). These models combine the power of transformers, transfer learning, and self-supervised learning (SSL…
Self-Supervised LearningSurveyTransfer Learning