Conformal Prediction: A Theoretical Note and Benchmarking Transductive Node Classification in Graphs
Conformal prediction has become increasingly popular for quantifying the uncertainty associated with machine learning models. Recent work in graph uncertainty quantification has built upon this approach for conformal graph prediction. The nascent nature of these explorations has led to conflicting choices for implementations, baselines, and method evaluation. In this work, we analyze the design choices made in the literature and discuss the tradeoffs associated with existing methods. Building on the existing implementations, we introduce techniques to scale existing methods to large-scale graph datasets without sacrificing performance. Our theoretical and empirical results justify our recommendations for future scholarship in graph conformal prediction.
Code (1)
Tasks
BenchmarkingConformal PredictionNode ClassificationPredictionUncertainty QuantificationSimilar Papers 제목 키워드 기반
Transductive conformal inference with adaptive scores
Conformal inference is a fundamental and versatile tool that provides distribution-free guarantees for many machine learning tasks. We consider the transductive setting, where decisions are made on a test sample of $m$ n…
Novelty DetectionTransfer LearningConformal Prediction for Zero-Shot Models
Vision-language models pre-trained at large scale have shown unprecedented adaptability and generalization to downstream tasks. Although its discriminative potential has been widely explored, its reliability and uncertai…
Conformal PredictionPredictionTransfer LearningFundamental bounds on efficiency-confidence trade-off for transductive conformal prediction
Transductive conformal prediction addresses the simultaneous prediction for multiple data points. Given a desired confidence level, the objective is to construct a prediction set that includes the true outcomes with the …
Interpretable Multivariate Conformal Prediction with Fast Transductive Standardization
We propose a conformal prediction method for constructing tight simultaneous prediction intervals for multiple, potentially related, numerical outputs given a single input. This method can be combined with any multi-targ…
Multi-target regressionConformal Prediction Regions are Imprecise Highest Density Regions
Recently, Cella and Martin proved how, under an assumption called consonance, a credal set (i.e. a closed and convex set of probabilities) can be derived from the conformal transducer associated with transductive conform…
Conformal PredictionPrediction