Classifier comparison using precision
New proposed models are often compared to state-of-the-art using statistical significance testing. Literature is scarce for classifier comparison using metrics other than accuracy. We present a survey of statistical methods that can be used for classifier comparison using precision, accounting for inter-precision correlation arising from use of same dataset. Comparisons are made using per-class precision and methods presented to test global null hypothesis of an overall model comparison. Comparisons are extended to multiple multi-class classifiers and to models using cross validation or its variants. Partial Bayesian update to precision is introduced when population prevalence of a class is known. Applications to compare deep architectures are studied.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Regular Expressions for Fast-response COVID-19 Text Classification
Text classifiers are at the core of many NLP applications and use a variety of algorithmic approaches and software. This paper introduces infrastructure and methodologies for text classifiers based on large-scale regular…
ClassificationGeneral Classificationtext-classificationText ClassificationOn the Geometry of Receiver Operating Characteristic and Precision-Recall Curves
We study the geometry of Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves in binary classification problems. The key finding is that many of the most commonly used binary classification metrics ar…
Binary ClassificationX-ray Astronomical Point Sources Recognition Using Granular Binary-tree SVM
The study on point sources in astronomical images is of special importance, since most energetic celestial objects in the Universe exhibit a point-like appearance. An approach to recognize the point sources (PS) in the X…
Quantifying the Uncertainty of Precision Estimates for Rule based Text Classifiers
Rule based classifiers that use the presence and absence of key sub-strings to make classification decisions have a natural mechanism for quantifying the uncertainty of their precision. For a binary classifier, the key i…
Multiclass Burn Wound Image Classification Using Deep Convolutional Neural Networks
Millions of people are affected by acute and chronic wounds yearly across the world. Continuous wound monitoring is important for wound specialists to allow more accurate diagnosis and optimization of management protocol…
ClassificationGeneral Classificationimage-classificationImage Classification+1