Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances. We conduct formative research with machine learning practitioners at Apple and find that conventional confusion matrices do not support more complex data-structures found in modern-day applications, such as hierarchical and multi-output labels. To express such variations of confusion matrices, we design an algebra that models confusion matrices as probability distributions. Based on this algebra, we develop Neo, a visual analytics system that enables practitioners to flexibly author and interact with hierarchical and multi-output confusion matrices, visualize derived metrics, renormalize confusions, and share matrix specifications. Finally, we demonstrate Neo's utility with three model evaluation scenarios that help people better understand model performance and reveal hidden confusions.
Code (1)
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Hierarchical confusion matrix for classification performance evaluation
In this work we propose a novel concept of a hierarchical confusion matrix, opening the door for popular confusion matrix based (flat) evaluation measures from binary classification problems, while considering the peculi…
Binary ClassificationClassificationConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion
Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data sc…
Active LearningModel SelectionNetwork PruningInequalities for Optimization of Classification Algorithms: A Perspective Motivated by Diagnostic Testing
Motivated by canonical problems in medical diagnostics, we propose and study properties of an objective function that uniformly bounds uncertainties in quantities of interest extracted from classifiers and related data a…
ClassSPLOM -- A Scatterplot Matrix to Visualize Separation of Multiclass Multidimensional Data
In multiclass classification of multidimensional data, the user wants to build a model of the classes to predict the label of unseen data. The model is trained on the data and tested on unseen data with known labels to e…
PyCM: Multiclass confusion matrix library in Python
PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall …
General Classification