Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks

2023-09-06 · Sanket Jantre, Nathan M. Urban, Xiaoning Qian, Byung-Jun Yoon

Bayesian inference for neural networks, or Bayesian deep learning, has the potential to provide well-calibrated predictions with quantified uncertainty and robustness. However, the main hurdle for Bayesian deep learning is its computational complexity due to the high dimensionality of the parameter space. In this work, we propose a novel scheme that addresses this limitation by constructing a low-dimensional subspace of the neural network parameters-referred to as an active subspace-by identifying the parameter directions that have the most significant influence on the output of the neural network. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference via either Monte Carlo (MC) sampling methods, otherwise computationally intractable, or variational inference. Empirically, our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.

📄 PDF Abstract BibTeX arXiv:2309.03061

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian InferenceDeep LearningUncertainty QuantificationVariational Inference

Deep active subspaces - a scalable method for high-dimensional uncertainty propagation

2019-02-27 · Rohit Tripathy, Ilias Bilionis

A problem of considerable importance within the field of uncertainty quantification (UQ) is the development of efficient methods for the construction of accurate surrogate models. Such efforts are particularly important …

Dimensionality Reduction Uncertainty Quantification Vocal Bursts Intensity Prediction

Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation

2022-09-20 · Thomas Buddenkotte, Lorena Escudero Sanchez, Mireia Crispin-Ortuzar, Ramona Woitek 외

Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, q…

Active Learning Uncertainty Quantification

Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design

2024-04-30 · A N M Nafiz Abeer, Sanket Jantre, Nathan M Urban, Byung-Jun Yoon

Deep generative models have been accelerating the inverse design process in material and drug design. Unlike their counterpart property predictors in typical molecular design frameworks, generative molecular design model…

Bayesian Inference Diversity Drug Design Uncertainty Quantification

Combining additivity and active subspaces for high-dimensional Gaussian process modeling

2024-02-06 · Mickael Binois, Victor Picheny

Gaussian processes are a widely embraced technique for regression and classification due to their good prediction accuracy, analytical tractability and built-in capabilities for uncertainty quantification. However, they …

Gaussian Processes Uncertainty Quantification

Bayesian Fine-tuning in Projected Subspaces

2026-05-08 · Viktar Dubovik, Patryk Marszałek, Jacek Tabor, Tomasz Kuśmierczyk arxiv

Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of large models by decomposing weight updates into low-rank matrices, significantly reducing storage and computational overhead. While effective, standar…

parameter-efficient fine-tuning Computational Efficiency