Uncertainty-aware Active Learning for Optimal Bayesian Classifier
For pool-based active learning, in each iteration a candidate training sample is chosen for labeling by optimizing an acquisition function. Expected Loss Reduction~(ELR) methods maximize the expected reduction in the classification error given a new labeled candidate based on a one-step-look-ahead strategy. ELR is the optimal strategy with a single query; however, since such myopic strategies cannot identify the long-term effect of a query on the classification error, ELR may get stuck before reaching the optimal classifier. To improve convergence in the context of optimal Bayesian classification, we utilize an one-step-look-ahead acquisition function based on a weighted form of the mean objective cost of uncertainty (MOCU) that focuses on the uncertainty related to the classification error. The weight scheme is designed to guarantee the Weighted-MOCU algorithm converging to the optimal classifier of the true model. We demonstrate its performance with both synthetic and real-world datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningClassificationGeneral ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Active Bayesian Assessment for Black-Box Classifiers
Recent advances in machine learning have led to increased deployment of black-box classifiers across a wide variety of applications. In many such situations there is a critical need to both reliably assess the performanc…
text-classificationText ClassificationUncertainty-Aware Autonomous Vehicles: Predicting the Road Ahead
Autonomous Vehicle (AV) perception systems have advanced rapidly in recent years, providing vehicles with the ability to accurately interpret their environment. Perception systems remain susceptible to errors caused by o…
Autonomous VehiclesAutonomous DrivingRobust Bayesian Decision Making under Adversarial Uncertainty
Scientific experiments are often designed to maximize information gain, yet in many applications the primary objective is to support reliable downstream decision-making. Existing decision-aware experimental design and ac…
Active LearningDecision MakingMultivariate Bayesian Last Layer for Regression: Uncertainty Quantification and Disentanglement
We present new Bayesian Last Layer models in the setting of multivariate regression under heteroscedastic noise, and propose an optimization algorithm for parameter learning. Bayesian Last Layer combines Bayesian modelli…
DisentanglementregressionUncertainty QuantificationBig Batch Bayesian Active Learning by Considering Predictive Probabilities
We observe that BatchBALD, a popular acquisition function for batch Bayesian active learning for classification, can conflate epistemic and aleatoric uncertainty, leading to suboptimal performance. Motivated by this obse…
Active Learning