The MCC-F1 curve: a performance evaluation technique for binary classification
Many fields use the ROC curve and the PR curve as standard evaluations of binary classification methods. Analysis of ROC and PR, however, often gives misleading and inflated performance evaluations, especially with an imbalanced ground truth. Here, we demonstrate the problems with ROC and PR analysis through simulations, and propose the MCC-F1 curve to address these drawbacks. The MCC-F1 curve combines two informative single-threshold metrics, MCC and the F1 score. The MCC-F1 curve more clearly differentiates good and bad classifiers, even with imbalanced ground truths. We also introduce the MCC-F1 metric, which provides a single value that integrates many aspects of classifier performance across the whole range of classification thresholds. Finally, we provide an R package that plots MCC-F1 curves and calculates related metrics.
Code (3)
Tasks
Binary ClassificationGeneral ClassificationSimilar Papers 제목 키워드 기반
A Study imbalance handling by various data sampling methods in binary classification
The purpose of this research report is to present the our learning curve and the exposure to the Machine Learning life cycle, with the use of a Kaggle binary classification data set and taking to explore various techniqu…
BIG-bench Machine LearningBinary ClassificationAutomatic classification of eclipsing binary stars using deep learning methods
In the last couple of decades, tremendous progress has been achieved in developing robotic telescopes and, as a result, sky surveys (both terrestrial and space) have become the source of a substantial amount of new obser…
ClassificationDeep LearningRotation Invariant Descriptors for Galaxy Morphological Classification
The detection of objects that are multi-oriented is a difficult pattern recognition problem. In this paper, we propose to evaluate the performance of different families of descriptors for the classification of galaxy mor…
Binary ClassificationClassificationGeneral ClassificationImage Retrieval+1Area under the ROC Curve has the Most Consistent Evaluation for Binary Classification
The proper use of model evaluation metrics is important for model evaluation and model selection in binary classification tasks. This study investigates how consistent different metrics are at evaluating models across da…
AllBinary ClassificationClassificationModel SelectionTopological Classification of points in $Z^2$ by using Topological Numbers for $2$D discrete binary images
In this paper, we propose a topological classification of points for 2D discrete binary images. This classification is based on the values of the calculus of topological numbers. Six classes of points are proposed: isola…
Classification