Conformal online model aggregation

2024-03-22 · Matteo Gasparin, Aaditya Ramdas

Conformal prediction equips machine learning models with a reasonable notion of uncertainty quantification without making strong distributional assumptions. It wraps around any black-box prediction model and converts point predictions into set predictions that have a predefined marginal coverage guarantee. However, conformal prediction only works if we fix the underlying machine learning model in advance. A relatively unaddressed issue in conformal prediction is that of model selection and/or aggregation: for a given problem, which of the plethora of prediction methods (random forests, neural nets, regularized linear models, etc.) should we conformalize? This paper proposes a new approach towards conformal model aggregation in online settings that is based on combining the prediction sets from several algorithms by voting, where weights on the models are adapted over time based on past performance.

📄 PDF Abstract BibTeX arXiv:2403.15527

Code (1)

matteogaspa/coma 공식 구현

Tasks

Conformal PredictionmodelModel SelectionPredictionUncertainty Quantification

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Online Localized Conformal Prediction

2026-05-06 · Yuheng Lai, Garvesh Raskutti arxiv

Conformal prediction is a framework that provides valid uncertainty quantification for general models with exchangeable data. However, in the online learning and time-series settings, exchangeability is not satisfied. Ex…

Adaptive Bayesian Online Learning via Expert Aggregation

2026-07-22 · Jungbin Jun, Ilsang Ohn arxiv

Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually…

Adaptive Conformal Predictions for Time Series

2022-02-15 · Margaux Zaffran, Aymeric Dieuleveut, Olivier Féron, Yannig Goude 외

Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. Whi…

Conformal Prediction Decision Making Prediction Intervals Time Series+2

CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators

2024-05-27 · Harry Zhang, Luca Carlone

We introduce CHAMP, a novel method for learning sequence-to-sequence, multi-hypothesis 3D human poses from 2D keypoints by leveraging a conditional distribution with a diffusion model. To predict a single output 3D pose …

Conformal Prediction Prediction

Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation

2024-05-25 · Eduardo Ochoa Rivera, Yash Patel, Ambuj Tewari

Distribution-free uncertainty estimation for ensemble methods is increasingly desirable due to the widening deployment of multi-modal black-box predictive models. Conformal prediction is one approach that avoids such dis…

Conformal Prediction MULTI-VIEW LEARNING Prediction Uncertainty Quantification