Marginal and training-conditional guarantees in one-shot federated conformal prediction
We study conformal prediction in the one-shot federated learning setting. The main goal is to compute marginally and training-conditionally valid prediction sets, at the server-level, in only one round of communication between the agents and the server. Using the quantile-of-quantiles family of estimators and split conformal prediction, we introduce a collection of computationally-efficient and distribution-free algorithms that satisfy the aforementioned requirements. Our approaches come from theoretical results related to order statistics and the analysis of the Beta-Beta distribution. We also prove upper bounds on the coverage of all proposed algorithms when the nonconformity scores are almost surely distinct. For algorithms with training-conditional guarantees, these bounds are of the same order of magnitude as those of the centralized case. Remarkably, this implies that the one-shot federated learning setting entails no significant loss compared to the centralized case. Our experiments confirm that our algorithms return prediction sets with coverage and length similar to those obtained in a centralized setting.
Code (1)
Tasks
Conformal PredictionFederated LearningPredictionvalidSimilar Papers 제목 키워드 기반
Federated Conditional Conformal Prediction via Generative Models
Conformal Prediction (CP) provides distribution-free uncertainty quantification by constructing prediction sets that guarantee coverage of the true labels. This reliability makes CP valuable for high-stakes federated lea…
Federated LearningMulti-Agent Conformal Prediction with Personalized Statistical Validity
Uncertainty quantification is essential in high-stakes machine learning tasks. However, one of the principled solutions, conformal prediction, faces challenges under limited local calibration data, privacy constraints, a…
Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction
Conformal prediction endows arbitrary black-box predictors with finite-sample, distribution-free marginal coverage, yet marginal validity can hide severe covariate-specific miscalibration, while exact distribution-free c…
A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities
Hate speech online remains an understudied issue for marginalized communities, and has seen rising relevance, especially in the Global South, which includes developing societies with increasing internet penetration. In t…
Federated LearningFew-Shot LearningHate Speech DetectionPrivacy PreservingPractical One-Shot Federated Learning for Cross-Silo Setting
Federated learning enables multiple parties to collaboratively learn a model without exchanging their data. While most existing federated learning algorithms need many rounds to converge, one-shot federated learning (i.e…
Federated LearningTransfer Learning