Performance measures for classification systems with rejection
Classifiers with rejection are essential in real-world applications where misclassifications and their effects are critical. However, if no problem specific cost function is defined, there are no established measures to assess the performance of such classifiers. We introduce a set of desired properties for performance measures for classifiers with rejection, based on which we propose a set of three performance measures for the evaluation of the performance of classifiers with rejection that satisfy the desired properties. The nonrejected accuracy measures the ability of the classifier to accurately classify nonrejected samples; the classification quality measures the correct decision making of the classifier with rejector; and the rejection quality measures the ability to concentrate all misclassified samples onto the set of rejected samples. From the measures, we derive the concept of relative optimality that allows us to connect the measures to a family of cost functions that take into account the trade-off between rejection and misclassification. We illustrate the use of the proposed performance measures on classifiers with rejection applied to synthetic and real-world data.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDecision MakingGeneral ClassificationSimilar Papers 제목 키워드 기반
Countermeasures Against Adversarial Examples in Radio Signal Classification
Deep learning algorithms have been shown to be powerful in many communication network design problems, including that in automatic modulation classification. However, they are vulnerable to carefully crafted attacks call…
ClassificationDeep LearningOptimal Rejection Function Meets Character Recognition Tasks
In this paper, we propose an optimal rejection method for rejecting ambiguous samples by a rejection function. This rejection function is trained together with a classification function under the framework of Learning-wi…
ClassificationLearning TheoryRobust hyperspectral image classification with rejection fields
In this paper we present a novel method for robust hyperspectral image classification using context and rejection. Hyperspectral image classification is generally an ill-posed image problem where pixels may belong to unk…
ClassificationGeneral ClassificationHyperspectral Image Classificationimage-classification+2Image Classification with Rejection using Contextual Information
We introduce a new supervised algorithm for image classification with rejection using multiscale contextual information. Rejection is desired in image-classification applications that require a robust classifier but not …
ClassificationGeneral Classificationimage-classificationImage ClassificationAn ASR-Based Tutor for Learning to Read: How to Optimize Feedback to First Graders
The interest in employing automatic speech recognition (ASR) in applications for reading practice has been growing in recent years. In a previous study, we presented an ASR-based Dutch reading tutor application that was …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition