Hierarchical confusion matrix for classification performance evaluation
In this work we propose a novel concept of a hierarchical confusion matrix, opening the door for popular confusion matrix based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. We develop the concept to a generalized form and prove its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi path labelling, and non mandatory leaf node prediction. Finally, we use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications and compare the results to established evaluation measures. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems. The implementation of hierarchical confusion matrix is available on GitHub.
Code (1)
Tasks
Binary ClassificationClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We condu…
BIG-bench Machine LearningPyCM: Multiclass confusion matrix library in Python
PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall …
General ClassificationAnyLoss: Transforming Classification Metrics into Loss Functions
Many evaluation metrics can be used to assess the performance of models in binary classification tasks. However, most of them are derived from a confusion matrix in a non-differentiable form, making it very difficult to …
Binary ClassificationClassificationModel SelectionOn multi-class learning through the minimization of the confusion matrix norm
In imbalanced multi-class classification problems, the misclassification rate as an error measure may not be a relevant choice. Several methods have been developed where the performance measure retained richer informatio…
General Classificationimbalanced classificationMulti-class ClassificationPAC-Bayesian Generalization Bound on Confusion Matrix for Multi-Class Classification
In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classif…
General ClassificationMulti-class Classification