Information-theoretic Analysis of Test Data Sensitivity in Uncertainty
Bayesian inference is often utilized for uncertainty quantification tasks. A recent analysis by Xu and Raginsky 2022 rigorously decomposed the predictive uncertainty in Bayesian inference into two uncertainties, called aleatoric and epistemic uncertainties, which represent the inherent randomness in the data-generating process and the variability due to insufficient data, respectively. They analyzed those uncertainties in an information-theoretic way, assuming that the model is well-specified and treating the model's parameters as latent variables. However, the existing information-theoretic analysis of uncertainty cannot explain the widely believed property of uncertainty, known as the sensitivity between the test and training data. It implies that when test data are similar to training data in some sense, the epistemic uncertainty should become small. In this work, we study such uncertainty sensitivity using our novel decomposition method for the predictive uncertainty. Our analysis successfully defines such sensitivity using information-theoretic quantities. Furthermore, we extend the existing analysis of Bayesian meta-learning and show the novel sensitivities among tasks for the first time.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceMeta-LearningSensitivityUncertainty QuantificationSimilar Papers 제목 키워드 기반
MultiContrievers: Analysis of Dense Retrieval Representations
Dense retrievers compress source documents into (possibly lossy) vector representations, yet there is little analysis of what information is lost versus preserved, and how it affects downstream tasks. We conduct the firs…
RetrievalSensitivityPhysical Sensitivity Kernels Can Emerge in Data-Driven Forward Models: Evidence From Surface-Wave Dispersion
Data-driven neural networks are increasingly used as surrogate forward models in geophysics, but it remains unclear whether they recover only the data mapping or also the underlying physical sensitivity structure. Here w…
Stochastic Sensitivity Analysis Using Fuzzy Influence Diagrams
The practice of stochastic sensitivity analysis described in the decision analysis literature is a testimonial to the need for considering deviations from precise point estimates of uncertainty. We propose the use of Bay…
Decision MakingDiagnosticSensitivityTesting predictions of representation cost theory with CNNs
It is widely acknowledged that trained convolutional neural networks (CNNs) have different levels of sensitivity to signals of different frequency. In particular, a number of empirical studies have documented CNNs sensit…
SensitivityPEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment
Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences. However, its adaptation to multimodal settings remains unexplored. Through repre…