Bayesian Active Learning With Abstention Feedbacks
We study pool-based active learning with abstention feedbacks where a labeler can abstain from labeling a queried example with some unknown abstention rate. This is an important problem with many useful applications. We take a Bayesian approach to the problem and develop two new greedy algorithms that learn both the classification problem and the unknown abstention rate at the same time. These are achieved by simply incorporating the estimated average abstention rate into the greedy criteria. We prove that both algorithms have near-optimality guarantees: they respectively achieve a ${(1-\frac{1}{e})}$ constant factor approximation of the optimal expected or worst-case value of a useful utility function. Our experiments show the algorithms perform well in various practical scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
Bayesian Pool-based Active Learning With Abstention Feedbacks
We study pool-based active learning with abstention feedbacks, where a labeler can abstain from labeling a queried example with some unknown abstention rate. This is an important problem with many useful applications. We…
Active LearningGeneral ClassificationBayesian Best-Arm Identification with Abstention: A Polynomial-to-Exponential Phase Transition
We study the Bayesian fixed-budget best-arm identification problem in which a learner can abstain from making a terminal recommendation. Subject to an abstention budget $α$, we analyze the probability of undetected error…
Active Learning for Binary Classification with Abstention
We construct and analyze active learning algorithms for the problem of binary classification with abstention. We consider three abstention settings: \emph{fixed-cost} and two variants of \emph{bounded-rate} abstention, a…
Active LearningBinary ClassificationClassificationGeneral ClassificationActive Learning from Imperfect Labelers
We study active learning where the labeler can not only return incorrect labels but also abstain from labeling. We consider different noise and abstention conditions of the labeler. We propose an algorithm which utilizes…
Active LearningEfficient Active Learning with Abstention
The goal of active learning is to achieve the same accuracy achievable by passive learning, while using much fewer labels. Exponential savings in terms of label complexity have been proved in very special cases, but fund…
Active Learning