MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification
We introduce MultiMatch, a novel semi-supervised learning (SSL) algorithm combining the paradigms of co-training and consistency regularization with pseudo-labeling. At its core, MultiMatch features a three-fold pseudo-label weighting module designed for three key purposes: selecting and filtering pseudo-labels based on head agreement and model confidence, and weighting them according to the perceived classification difficulty. This novel module enhances and unifies three existing techniques -- heads agreement from Multihead Co-training, self-adaptive thresholds from FreeMatch, and Average Pseudo-Margins from MarginMatch -- resulting in a holistic approach that improves robustness and performance in SSL settings. Experimental results on benchmark datasets highlight the superior performance of MultiMatch, achieving state-of-the-art results on 9 out of 10 setups from 5 natural language processing datasets and ranking first according to the Friedman test among 19 methods. Furthermore, MultiMatch demonstrates exceptional robustness in highly imbalanced settings, outperforming the second-best approach by 3.26% -- and data imbalance is a key factor for many text classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Pseudo LabelSemi-Supervised Text Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Semi-DETR: Semi-Supervised Object Detection with Detection Transformers
We analyze the DETR-based framework on semi-supervised object detection (SSOD) and observe that (1) the one-to-one assignment strategy generates incorrect matching when the pseudo ground-truth bounding box is inaccurate,…
Objectobject-detectionObject DetectionPseudo Label+1AstMatch: Adversarial Self-training Consistency Framework for Semi-Supervised Medical Image Segmentation
Semi-supervised learning (SSL) has shown considerable potential in medical image segmentation, primarily leveraging consistency regularization and pseudo-labeling. However, many SSL approaches only pay attention to low-l…
Image SegmentationMedical Image SegmentationPseudo LabelSemantic Segmentation+2MultiMatch: Multi-task Learning for Semi-supervised Domain Generalization
Domain generalization (DG) aims at learning a model on source domains to well generalize on the unseen target domain. Although it has achieved great success, most of existing methods require the label information for all…
Domain GeneralizationMulti-Task LearningPseudo LabelSemi-Supervised Domain GeneralizationEfficient PINNs: Multi-Head Unimodular Regularization of the Solutions Space
We present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based …
Transfer LearningRegion-level Contrastive and Consistency Learning for Semi-Supervised Semantic Segmentation
Current semi-supervised semantic segmentation methods mainly focus on designing pixel-level consistency and contrastive regularization. However, pixel-level regularization is sensitive to noise from pixels with incorrect…
SegmentationSemantic SegmentationSemi-Supervised Semantic Segmentation