Active Learning from Imperfect Labelers
We study active learning where the labeler can not only return incorrect labels but also abstain from labeling. We consider different noise and abstention conditions of the labeler. We propose an algorithm which utilizes abstention responses, and analyze its statistical consistency and query complexity under fairly natural assumptions on the noise and abstention rate of the labeler. This algorithm is adaptive in a sense that it can automatically request less queries with a more informed or less noisy labeler. We couple our algorithm with lower bounds to show that under some technical conditions, it achieves nearly optimal query complexity.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningSimilar Papers 제목 키워드 기반
Optimal Labeler Assignment and Sampling for Active Learning in the Presence of Imperfect Labels
Active Learning (AL) has garnered significant interest across various application domains where labeling training data is costly. AL provides a framework that helps practitioners query informative samples for annotation …
Active LearningImproved Adaptive Algorithm for Scalable Active Learning with Weak Labeler
Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions provided by weak labelers. We study this p…
Active LearningDSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation
Image segmentation is one of the most essential biomedical image processing problems for different imaging modalities, including microscopy and X-ray in the Internet-of-Medical-Things (IoMT) domain. However, annotating b…
Active LearningEnsemble LearningImage SegmentationSemantic SegmentationCost-Accuracy Aware Adaptive Labeling for Active Learning
Conventional active learning algorithms assume a single labeler that produces noiseless label at a given, fixed cost, and aim to achieve the best generalization performance for given classifier under a budget constraint.…
Active LearningInformativenessCost-Quality Adaptive Active Learning for Chinese Clinical Named Entity Recognition
Clinical Named Entity Recognition (CNER) aims to automatically identity clinical terminologies in Electronic Health Records (EHRs), which is a fundamental and crucial step for clinical research. To train a high-performan…
Active LearningInformativenessnamed-entity-recognitionNamed Entity Recognition+2