Confusion Matrices and Accuracy Statistics for Binary Classifiers Using Unlabeled Data: The Diagnostic Test Approach
Medical researchers have solved the problem of estimating the sensitivity and specificity of binary medical diagnostic tests without gold standard tests for comparison. That problem is the same as estimating confusion matrices for classifiers on unlabeled data. This article describes how to modify the diagnostic test solutions to estimate confusion matrices and accuracy statistics for supervised or unsupervised binary classifiers on unlabeled data.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticSensitivitySpecificityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PyCM: Multiclass confusion matrix library in Python
PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall …
General ClassificationFairness and Unfairness in Binary and Multiclass Classification: Quantifying, Calculating, and Bounding
We propose a new interpretable measure of unfairness, that allows providing a quantitative analysis of classifier fairness, beyond a dichotomous fair/unfair distinction. We show how this measure can be calculated when th…
FairnessScheduling to Learn In An Unsupervised Online Streaming Model
An unsupervised online streaming model is considered where samples arrive in an online fashion over $T$ slots. There are $M$ classifiers, whose confusion matrices are unknown a priori. In each slot, at most one sample ca…
SchedulingConfusion matrices and rough set data analysis
A widespread approach in machine learning to evaluate the quality of a classifier is to cross -- classify predicted and actual decision classes in a confusion matrix, also called error matrix. A classification tool which…
BIG-bench Machine LearningGeneral ClassificationBridging the Gap: Unifying the Training and Evaluation of Neural Network Binary Classifiers
While neural network binary classifiers are often evaluated on metrics such as Accuracy and $F_1$-Score, they are commonly trained with a cross-entropy objective. How can this training-evaluation gap be addressed? While …
General Classification