What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization

2022-07-11 · NeurIPS 2023 11 · Hao Sun, Boris van Breugel, Jonathan Crabbe, Nabeel Seedat, Mihaela van der Schaar

Uncertainty Quantification (UQ) is essential for creating trustworthy machine learning models. Recent years have seen a steep rise in UQ methods that can flag suspicious examples, however, it is often unclear what exactly these methods identify. In this work, we propose a framework for categorizing uncertain examples flagged by UQ methods in classification tasks. We introduce the confusion density matrix -- a kernel-based approximation of the misclassification density -- and use this to categorize suspicious examples identified by a given uncertainty method into three classes: out-of-distribution (OOD) examples, boundary (Bnd) examples, and examples in regions of high in-distribution misclassification (IDM). Through extensive experiments, we show that our framework provides a new and distinct perspective for assessing differences between uncertainty quantification methods, thereby forming a valuable assessment benchmark.

📄 PDF Abstract BibTeX arXiv:2207.05161

Code (0)

등록된 구현이 없습니다.

Tasks

Uncertainty Quantification

Uncertainty-aware Probabilistic 3D Human Motion Forecasting via Invertible Networks

2025-07-19 · Yue Ma, Kanglei Zhou, Fuyang Yu, Frederick W. B. Li 외 arxiv

3D human motion forecasting aims to enable autonomous applications. Estimating uncertainty for each prediction (i.e., confidence based on probability density or quantile) is essential for safety-critical contexts like hu…

Motion Forecasting Decision Making

Density Regression and Uncertainty Quantification with Bayesian Deep Noise Neural Networks

2022-06-12 · Daiwei Zhang, Tianci Liu, Jian Kang

Deep neural network (DNN) models have achieved state-of-the-art predictive accuracy in a wide range of supervised learning applications. However, accurately quantifying the uncertainty in DNN predictions remains a challe…

regression Uncertainty Quantification

Investigating and Improving Latent Density Segmentation Models for Aleatoric Uncertainty Quantification in Medical Imaging

2023-07-31 · M. M. Amaan Valiuddin, Christiaan G. A. Viviers, Ruud J. G. van Sloun, Peter H. N. de With 외

Data uncertainties, such as sensor noise, occlusions or limitations in the acquisition method can introduce irreducible ambiguities in images, which result in varying, yet plausible, semantic hypotheses. In Machine Learn…

Image Segmentation Informativeness Medical Image Segmentation Segmentation+2

Scalable Posterior Uncertainty for Flexible Density-Based Clustering

2026-03-03 · Nicola Bariletto, Stephen G. Walker arxiv

We introduce a novel framework for uncertainty quantification in clustering that combines martingale posterior distributions with density-based clustering. Unlike classical model-based approaches, which define clusters a…

Computational Efficiency

Improved Uncertainty Estimation of Graph Neural Network Potentials Using Engineered Latent Space Distances

2024-07-15 · Joseph Musielewicz, Janice Lan, Matt Uyttendaele, John R. Kitchin

Graph neural networks (GNNs) have been shown to be astonishingly capable models for molecular property prediction, particularly as surrogates for expensive density functional theory calculations of relaxed energy for nov…

Graph Neural Network Molecular Property Prediction Prediction Property Prediction+1