Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation
Distribution-free uncertainty estimation for ensemble methods is increasingly desirable due to the widening deployment of multi-modal black-box predictive models. Conformal prediction is one approach that avoids such distributional assumptions. Methods for conformal aggregation have in turn been proposed for ensembled prediction, where the prediction regions of individual models are merged as to retain coverage guarantees while minimizing conservatism. Merging the prediction regions directly, however, sacrifices structures present in the conformal scores that can further reduce conservatism. We, therefore, propose a novel framework that extends the standard scalar formulation of a score function to a multivariate score that produces more efficient prediction regions. We then demonstrate that such a framework can be efficiently leveraged in both classification and predict-then-optimize regression settings downstream and empirically show the advantage over alternate conformal aggregation methods.
Code (1)
Tasks
Conformal PredictionMULTI-VIEW LEARNINGPredictionUncertainty QuantificationSimilar Papers 제목 키워드 기반
Weighted Aggregation of Conformity Scores for Classification
Conformal prediction is a powerful framework for constructing prediction sets with valid coverage guarantees in multi-class classification. However, existing methods often rely on a single score function, which can limit…
ClassificationConformal PredictionInformativenessMulti-class Classification+2Distribution-Aware Conformal Prediction: A Framework for generating efficient prediction intervals for time series
We present Distribution-aware Conformal Prediction (DCP), a unified framework integrating probabilistic predictors like Monte Carlo dropout, deep ensembles, and quantile regression with score-agnostic conformal calibrati…
Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets
Access to multiple predictive models trained for the same task, whether in regression or classification, is increasingly common in many applications. Aggregating their predictive uncertainties to produce reliable and eff…
CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators
We introduce CHAMP, a novel method for learning sequence-to-sequence, multi-hypothesis 3D human poses from 2D keypoints by leveraging a conditional distribution with a diffusion model. To predict a single output 3D pose …
Conformal PredictionPredictionAggregating Conformal Prediction Sets via α-Allocation
Conformal prediction offers a distribution-free framework for constructing prediction sets with finite-sample coverage. Yet, efficiently leveraging multiple conformity scores to reduce prediction set size remains a major…