Aggregating Predictions on Multiple Non-disclosed Datasets using Conformal Prediction
Conformal Prediction is a machine learning methodology that produces valid prediction regions under mild conditions. In this paper, we explore the application of making predictions over multiple data sources of different sizes without disclosing data between the sources. We propose that each data source applies a transductive conformal predictor independently using the local data, and that the individual predictions are then aggregated to form a combined prediction region. We demonstrate the method on several data sets, and show that the proposed method produces conservatively valid predictions and reduces the variance in the aggregated predictions. We also study the effect that the number of data sources and size of each source has on aggregated predictions, as compared with equally sized sources and pooled data.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionPredictionvalidSimilar Papers 제목 키워드 기반
Combining Prediction Intervals on Multi-Source Non-Disclosed Regression Datasets
Conformal Prediction is a framework that produces prediction intervals based on the output from a machine learning algorithm. In this paper we explore the case when training data is made up of multiple parts available in…
BIG-bench Machine LearningConformal PredictionPredictionPrediction Intervals+2Aggregation in conformal e-classification
Aggregating conformal predictors is a standard way of balancing their predictive and computational efficiency while retaining their validity, at least approximately. An important advantage of conformal e-predictors is th…
Computational EfficiencyAggregating Conformal Prediction Sets via α-Allocation
Conformal prediction offers a distribution-free framework for constructing prediction sets with finite-sample coverage. Yet, efficiently leveraging multiple conformity scores to reduce prediction set size remains a major…
Spatial-Aware Conformal Prediction for Trustworthy Hyperspectral Image Classification
Hyperspectral image (HSI) classification involves assigning unique labels to each pixel to identify various land cover categories. While deep classifiers have achieved high predictive accuracy in this field, they lack th…
ClassificationConformal PredictionHyperspectral Image Classificationimage-classification+3Conformal Set-based Human-AI Complementarity with Multiple Experts
Decision support systems are designed to assist human experts in classification tasks by providing conformal prediction sets derived from a pre-trained model. This human-AI collaboration has demonstrated enhanced classif…