Adaptive calibration for binary classification
This note proposes a way of making probability forecasting rules less sensitive to changes in data distribution, concentrating on the simple case of binary classification. This is important in applications of machine learning, where the quality of a trained predictor may drop significantly in the process of its exploitation. Our techniques are based on recent work on conformal test martingales and older work on prediction with expert advice, namely tracking the best expert.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationClassificationSimilar Papers 제목 키워드 기반
Model Calibration in Dense Classification with Adaptive Label Perturbation
For safety-related applications, it is crucial to produce trustworthy deep neural networks whose prediction is associated with confidence that can represent the likelihood of correctness for subsequent decision-making. E…
Binary ClassificationClassificationDecision MakingPredictionCalibrating Deep Neural Networks by Pairwise Constraints
It is well known that deep neural networks (DNNs) produce poorly calibrated estimates of class-posterior probabilities. We hypothesize that this is due to the limited calibration supervision provided by the cross-ent…
Binary ClassificationOn the Calibration of Nested Dichotomies for Large Multiclass Tasks
Nested dichotomies are used as a method of transforming a multiclass classification problem into a series of binary problems. A tree structure is induced that recursively splits the set of classes into subsets, and a bin…
Binary ClassificationGeneral ClassificationAdaptive Semi-Supervised Training of P300 ERP-BCI Speller System with Minimum Calibration Effort
A P300 ERP-based Brain-Computer Interface (BCI) speller is an assistive communication tool. It searches for the P300 event-related potential (ERP) elicited by target stimuli, distinguishing it from the neural responses t…
Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models
Learning accurate probabilistic models from data is crucial in many practical tasks in data mining. In this paper we present a new non-parametric calibration method called \textit{ensemble of near isotonic regression} (E…
Binary ClassificationClassifier calibrationGeneral Classificationregression