Confidence intervals for class prevalences under prior probability shift
Point estimation of class prevalences in the presence of data set shift has been a popular research topic for more than two decades. Less attention has been paid to the construction of confidence and prediction intervals for estimates of class prevalences. One little considered question is whether or not it is necessary for practical purposes to distinguish confidence and prediction intervals. Another question so far not yet conclusively answered is whether or not the discriminatory power of the classifier or score at the basis of an estimation method matters for the accuracy of the estimates of the class prevalences. This paper presents a simulation study aimed at shedding some light on these and other related questions.
Code (0)
등록된 구현이 없습니다.
Tasks
Prediction IntervalsSimilar Papers 제목 키워드 기반
Biomechanical Reconstruction with Confidence Intervals from Multiview Markerless Motion Capture
Advances in multiview markerless motion capture (MMMC) promise high-quality movement analysis for clinical practice and research. While prior validation studies show MMMC performs well on average, they do not provide wha…
Markerless Motion CaptureMixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals
This paper presents new deviation inequalities that are valid uniformly in time under adaptive sampling in a multi-armed bandit model. The deviations are measured using the Kullback-Leibler divergence in a given one-dime…
validWhen to Trust Confidence Thresholding: Calibration Diagnostics for Pseudo-Labelled Regression
Calibrated probability outputs of trained classifiers are increasingly used as inputs to downstream regression estimands such as effects, prevalences, or disparities for a latent group observed only on a small labelled s…
Minimax Confidence Intervals for the Sliced Wasserstein Distance
Motivated by the growing popularity of variants of the Wasserstein distance in statistics and machine learning, we study statistical inference for the Sliced Wasserstein distance--an easily computable variant of the Wass…
Uncertainty QuantificationConfidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects
We consider the construction of confidence intervals for treatment effects estimated using panel models with interactive fixed effects. We first use the factor-based matrix completion technique proposed by Bai and Ng (20…
Matrix Completion