Robust performance metrics for imbalanced classification problems
We show that established performance metrics in binary classification, such as the F-score, the Jaccard similarity coefficient or Matthews' correlation coefficient (MCC), are not robust to class imbalance in the sense that if the proportion of the minority class tends to $0$, the true positive rate (TPR) of the Bayes classifier under these metrics tends to $0$ as well. Thus, in imbalanced classification problems, these metrics favour classifiers which ignore the minority class. To alleviate this issue we introduce robust modifications of the F-score and the MCC for which, even in strongly imbalanced settings, the TPR is bounded away from $0$. We numerically illustrate the behaviour of the various performance metrics in simulations as well as on a credit default data set. We also discuss connections to the ROC and precision-recall curves and give recommendations on how to combine their usage with performance metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationClassificationimbalanced classificationSimilar Papers 제목 키워드 기반
$F_β$-plot -- a visual tool for evaluating imbalanced data classifiers
One of the significant problems associated with imbalanced data classification is the lack of reliable metrics. This runs primarily from the fact that for most real-life (as well as commonly used benchmark) problems, we …
ClassificationExact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification
For classification with imbalanced class frequencies, i.e., imbalanced classification (IC), standard accuracy is known to be misleading as a performance measure. While most existing methods for IC resort to optimizing ba…
The MCC-F1 curve: a performance evaluation technique for binary classification
Many fields use the ROC curve and the PR curve as standard evaluations of binary classification methods. Analysis of ROC and PR, however, often gives misleading and inflated performance evaluations, especially with an im…
Binary ClassificationGeneral ClassificationCost-Sensitive Convolution based Neural Networks for Imbalanced Time-Series Classification
Some deep convolutional neural networks were proposed for time-series classification and class imbalanced problems. However, those models performed degraded and even failed to recognize the minority class of an imbalance…
Deep LearningGeneral ClassificationTemporal SequencesTime Series+2QCS-ADME: Quantum Circuit Search for Drug Property Prediction with Imbalanced Data and Regression Adaptation
The biomedical field is beginning to explore the use of quantum machine learning (QML) for tasks traditionally handled by classical machine learning, especially in predicting ADME (absorption, distribution, metabolism, a…
Classificationimbalanced classificationProperty PredictionQuantum Machine Learning+1