Binary Classification from Positive Data with Skewed Confidence
Positive-confidence (Pconf) classification [Ishida et al., 2018] is a promising weakly-supervised learning method which trains a binary classifier only from positive data equipped with confidence. However, in practice, the confidence may be skewed by bias arising in an annotation process. The Pconf classifier cannot be properly learned with skewed confidence, and consequently, the classification performance might be deteriorated. In this paper, we introduce the parameterized model of the skewed confidence, and propose the method for selecting the hyperparameter which cancels out the negative impact of skewed confidence under the assumption that we have the misclassification rate of positive samples as a prior knowledge. We demonstrate the effectiveness of the proposed method through a synthetic experiment with simple linear models and benchmark problems with neural network models. We also apply our method to drivers' drowsiness prediction to show that it works well with a real-world problem where confidence is obtained based on manual annotation.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationClassificationGeneral ClassificationWeakly-supervised LearningSimilar Papers 제목 키워드 기반
Classification from Positive and Biased Negative Data with Skewed Labeled Posterior Probability
The binary classification problem has a situation where only biased data are observed in one of the classes. In this paper, we propose a new method to approach the positive and biased negative (PbN) classification proble…
Binary ClassificationClassificationWeakly-supervised LearningBinary Classification from Positive-Confidence Data
Can we learn a binary classifier from only positive data, without any negative data or unlabeled data? We show that if one can equip positive data with confidence (positive-confidence), one can successfully learn a binar…
Binary ClassificationClassificationClusteringGeneral Classification+1Negative Confidence-Aware Weakly Supervised Binary Classification for Effective Review Helpfulness Classification
The incompleteness of positive labels and the presence of many unlabelled instances are common problems in binary classification applications such as in review helpfulness classification. Various studies from the classif…
Binary ClassificationClassificationGeneral ClassificationBinary Classification with Confidence Difference
Recently, learning with soft labels has been shown to achieve better performance than learning with hard labels in terms of model generalization, calibration, and robustness. However, collecting pointwise labeling confid…
Binary ClassificationClassificationRecommendation SystemsDon't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification
Modern machine learning models with high accuracy are often miscalibrated -- the predicted top probability does not reflect the actual accuracy, and tends to be over-confident. It is commonly believed that such over-conf…
Binary Classification