W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators
Evaluating the variability of posterior estimates is a key aspect of Bayesian model assessment. In this study, we focus on the posterior covariance matrix W, defined using the log-likelihood of each observation. Previous studies have examined the role of the principal space of W in Bayesian sensitivity analysis, notably MacEachern and Peruggia (2002) and Thomas et al. (2018). In this work, we show that the principal space of W is also relevant for frequentist evaluation, using the recently proposed Bayesian infinitesimal jackknife (IJ) approximation Giordano and Broderick (2023) as a key tool. We next consider the relationship between the matrix W and the Fisher kernel. We show that the Fisher kernel can be regarded as an approximation to W; the matrix W, in itself, can be interpreted as a reproducing kernel, which we refer to as the W-kernel. Based on this connection, we examine the dual relationship between the W-kernel formulation in the data space and the classical asymptotic formulation in the parameter space. These ideas suggest a form of Bayesian-frequentist duality that emerges through the dual structure of kernel PCA, where posterior and frequentist covariances serve as inner products in their respective spaces. As an application, we consider an approximate bootstrap of posterior means based on posterior samples generated by MCMC. We show that the projection onto the principal space of W facilitates frequentist evaluation, particularly of the higher-order term in this procedure. In one of the appendices, we introduce incomplete Cholesky decomposition as an efficient method for computing the principal space of W and discuss the related concept of representative subsets of the observations.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Dimensionality Reduction and State Space Systems: Forecasting the US Treasury Yields Using Frequentist and Bayesian VARs
Using a state-space system, I forecasted the US Treasury yields by employing frequentist and Bayesian methods after first decomposing the yields of varying maturities into its unobserved term structure factors. Then, I e…
Dimensionality ReductionVariable SelectionInformation Directed Sampling and Bandits with Heteroscedastic Noise
In the stochastic bandit problem, the goal is to maximize an unknown function via a sequence of noisy evaluations. Typically, the observation noise is assumed to be independent of the evaluation point and to satisfy a ta…
Bayesian OptimizationThompson SamplingQuasi-Bayesian Dual Instrumental Variable Regression
Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking. …
Bayesian InferenceregressionUncertainty QuantificationOn Information Gain and Regret Bounds in Gaussian Process Bandits
Consider the sequential optimization of an expensive to evaluate and possibly non-convex objective function $f$ from noisy feedback, that can be considered as a continuum-armed bandit problem. Upper bounds on the regret …
No-Regret Algorithms for Time-Varying Bayesian Optimization
In this paper, we consider the time-varying Bayesian optimization problem. The unknown function at each time is assumed to lie in an RKHS (reproducing kernel Hilbert space) with a bounded norm. We adopt the general varia…
Bayesian Optimization