Mirror Online Conformal Prediction with Intermittent Feedback
Online conformal prediction enables the runtime calibration of a pre-trained artificial intelligence model using feedback on its performance. Calibration is achieved through set predictions that are updated via online rules so as to ensure long-term coverage guarantees. While recent research has demonstrated the benefits of incorporating prior knowledge into the calibration process, this has come at the cost of replacing coverage guarantees with less tangible regret guarantees based on the quantile loss. This work introduces intermittent mirror online conformal prediction (IM-OCP), a novel runtime calibration framework that integrates prior knowledge, while maintaining long-term coverage and achieving sub-linear regret. IM-OCP features closed-form updates with minimal memory complexity, and is designed to operate under potentially intermittent feedback.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionPredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedback
In interactive imitation learning (IL), uncertainty quantification offers a way for the learner (i.e. robot) to contend with distribution shifts encountered during deployment by actively seeking additional feedback from …
Conformal PredictionImitation LearningPredictionPrediction Intervals+1Online Conformal Prediction with Adversarial Semi-bandit Feedback via Regret Minimization
Uncertainty quantification is crucial in safety-critical systems, where decisions must be made under uncertainty. In particular, we consider the problem of online uncertainty quantification, where data points arrive sequ…
Graph-Structured Feedback Multimodel Ensemble Online Conformal Prediction
Online conformal prediction has demonstrated its capability to construct a prediction set for each incoming data point that covers the true label with a predetermined probability. To cope with potential distribution shif…
Conformal PredictionPredictionStochastic Online Conformal Prediction with Semi-Bandit Feedback
Conformal prediction has emerged as an effective strategy for uncertainty quantification by modifying a model to output sets of labels instead of a single label. These prediction sets come with the guarantee that they co…
Conformal Predictionimage-classificationImage ClassificationPrediction+2Online Conformal Probabilistic Numerics via Adaptive Edge-Cloud Offloading
Consider an edge computing setting in which a user submits queries for the solution of a linear system to an edge processor, which is subject to time-varying computing availability. The edge processor applies a probabili…
Conformal PredictionEdge-computingPrediction