Anytime-valid, Bayes-assisted,Prediction-Powered Inference
Given a large pool of unlabelled data and a smaller amount of labels, prediction-powered inference (PPI) leverages machine learning predictions to increase the statistical efficiency of standard confidence interval procedures based solely on labelled data, while preserving their fixed-time validity. In this paper, we extend the PPI framework to the sequential setting, where labelled and unlabelled datasets grow over time. Exploiting Ville's inequality and the method of mixtures, we propose prediction-powered confidence sequence procedures that are valid uniformly over time and naturally accommodate prior knowledge on the quality of the predictions to further boost efficiency. We carefully illustrate the design choices behind our method and demonstrate its effectiveness in real and synthetic examples.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionvalidSimilar Papers 제목 키워드 기반
Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means
Confidence sequences based on test martingales provide time-uniform uncertainty quantification for the mean of bounded IID observations without parametric distributional assumptions. Their practical efficiency, however, …
FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference
Prediction-powered inference (PPI) enables valid statistical inference by combining experimental data with machine learning predictions. When a sufficient number of high-quality predictions is available, PPI results in m…
PredictionvalidPrediction-Powered E-Values
Quality statistical inference requires a sufficient amount of data, which can be missing or hard to obtain. To this end, prediction-powered inference has risen as a promising methodology, but existing approaches are larg…
Causal DiscoveryChange Point DetectionPredictionPrediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts
We study the problem of monitoring model performance in dynamic environments where labeled data are limited. To this end, we propose prediction-powered risk monitoring (PPRM), a semi-supervised risk-monitoring approach b…
Image ClassificationRobust Bayes-Assisted Conformal Prediction
Bayes-assisted conformal prediction combines the strengths of Bayesian modelling with exact, distribution-free frequentist coverage guarantees. Although conformal validity is preserved even when the Bayesian working mode…