Conformal testing in a binary model situation
Conformal testing is a way of testing the IID assumption based on conformal prediction. The topic of this note is computational evaluation of the performance of conformal testing in a model situation in which IID binary observations generated from a Bernoulli distribution are followed by IID binary observations generated from another Bernoulli distribution, with the parameters of the distributions and changepoint unknown. Existing conformal test martingales can be used for this task and work well in simple cases, but their efficiency can be improved greatly.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionmodelSimilar Papers 제목 키워드 기반
Conformal testing: binary case with Markov alternatives
We continue study of conformal testing in binary model situations. In this note we consider Markov alternatives to the null hypothesis of exchangeability. We propose two new classes of conformal test martingales; one cla…
Computational EfficiencyIntegrative conformal p-values for powerful out-of-distribution testing with labeled outliers
This paper develops novel conformal methods to test whether a new observation was sampled from the same distribution as a reference set. Blending inductive and transductive conformal inference in an innovative way, the d…
Conformal Selective Prediction with General Risk Control
In deploying artificial intelligence (AI) models, selective prediction offers the option to abstain from making a prediction when uncertain about model quality. To fulfill its promise, it is crucial to enforce strict and…
Drug DiscoveryMax-Rank: Efficient Multiple Testing for Conformal Prediction
Multiple hypothesis testing (MHT) frequently arises in scientific inquiries, and concurrent testing of multiple hypotheses inflates the risk of Type-I errors or false positives, rendering MHT corrections essential. This …
Conformal PredictionPredictionUncertainty QuantificationInductive randomness predictors
This paper introduces inductive randomness predictors, which form a superset of inductive conformal predictors. Its focus is on a very simple special case, binary inductive randomness predictors. It is interesting that b…