Binary Classification with Positive Labeling Sources
To create a large amount of training labels for machine learning models effectively and efficiently, researchers have turned to Weak Supervision (WS), which uses programmatic labeling sources rather than manual annotation. Existing works of WS for binary classification typically assume the presence of labeling sources that are able to assign both positive and negative labels to data in roughly balanced proportions. However, for many tasks of interest where there is a minority positive class, negative examples could be too diverse for developers to generate indicative labeling sources. Thus, in this work, we study the application of WS on binary classification tasks with positive labeling sources only. We propose WEAPO, a simple yet competitive WS method for producing training labels without negative labeling sources. On 10 benchmark datasets, we show WEAPO achieves the highest averaged performance in terms of both the quality of synthesized labels and the performance of the final classifier supervised with these labels. We incorporated the implementation of \method into WRENCH, an existing benchmarking platform.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingBinary ClassificationClassificationSimilar Papers 제목 키워드 기반
Adaptive Pseudo Label Selection for Individual Unlabeled Data by Positive and Unlabeled Learning
This paper proposes a novel pseudo-labeling method for medical image segmentation that can perform learning on ``individual images'' to select effective pseudo-labels. We introduce Positive and Unlabeled Learning (PU lea…
Medical Image SegmentationBinary ClassificationComparison of Multi-Class and Binary Classification Machine Learning Models in Identifying Strong Gravitational Lenses
Typically, binary classification lens-finding schemes are used to discriminate between lens candidates and non-lenses. However, these models often suffer from substantial false-positive classifications. Such false positi…
BIG-bench Machine LearningBinary ClassificationGeneral ClassificationMulti-class ClassificationBinary Classification with Confidence Difference
Recently, learning with soft labels has been shown to achieve better performance than learning with hard labels in terms of model generalization, calibration, and robustness. However, collecting pointwise labeling confid…
Binary ClassificationClassificationRecommendation SystemsAutomatic Debiased Learning from Positive, Unlabeled, and Exposure Data
We address the issue of binary classification from positive and unlabeled data (PU classification) with a selection bias in the positive data. During the observation process, (i) a sample is exposed to a user, (ii) the u…
Binary ClassificationRecommendation SystemsSelection biasLearning from Similarity/Dissimilarity and Pairwise Comparison
This paper addresses binary classification in scenarios where obtaining explicit instance level labels is impractical, by exploiting multiple weak labels defined on instance pairs. The existing SconfConfDiff classificati…
Binary Classification