Unbiased Estimating Equation on Inverse Divergence and Its Conditions
This paper focuses on the Bregman divergence defined by the reciprocal function, called the inverse divergence. For the loss function defined by the monotonically increasing function $f$ and inverse divergence, the conditions for the statistical model and function $f$ under which the estimating equation is unbiased are clarified. Specifically, we characterize two types of statistical models, an inverse Gaussian type and a mixture of generalized inverse Gaussian type distributions, to show that the conditions for the function $f$ are different for each model. We also define Bregman divergence as a linear sum over the dimensions of the inverse divergence and extend the results to the multi-dimensional case.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Unbiased Estimation Equation under $f$-Separable Bregman Distortion Measures
We discuss unbiased estimation equations in a class of objective function using a monotonically increasing function $f$ and Bregman divergence. The choice of the function $f$ gives desirable properties such as robustness…
Statistical Inference for High-Dimensional Linear Regression with Blockwise Missing Data
Blockwise missing data occurs frequently when we integrate multisource or multimodality data where different sources or modalities contain complementary information. In this paper, we consider a high-dimensional linear r…
ImputationregressionvalidEstimating 2-Sinkhorn Divergence between Gaussian Processes from Finite-Dimensional Marginals
\emph{Optimal Transport} (OT) has emerged as an important computational tool in machine learning and computer vision, providing a geometrical framework for studying probability measures. OT unfortunately suffers from the…
Gaussian ProcessesBOOSTING ENCODER-DECODER CNN FOR INVERSE PROBLEMS
Encoder-decoder convolutional neural networks (CNN) have been extensively used for various inverse problems. However, their prediction error for unseen test data is difficult to estimate a priori, since the neural …
DecoderDenoisingPredictionBetter Estimation of the KL Divergence Between Language Models
Estimating the Kullback--Leibler (KL) divergence between language models has many applications, e.g., reinforcement learning from human feedback (RLHF), interpretability, and knowledge distillation. However, computing th…
Knowledge Distillation