Minimum Relative Entropy Inference for Normal and Monte Carlo Distributions
We represent affine sub-manifolds of exponential family distributions as minimum relative entropy sub-manifolds. With such representation we derive analytical formulas for the inference from partial information on expectations and covariances of multivariate normal distributions; and we improve the numerical implementation via Monte Carlo simulations for the inference from partial information of generalized expectation type.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Entropic Causal Inference
We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the un…
Causal InferenceTesting for the Minimum Mean-Variance Spanning Set
This paper explores the estimation and inference of the minimum spanning set (MSS), the smallest subset of risky assets that spans the mean-variance efficient frontier of the full asset set. We establish identification c…
Relative Entropy, Probabilistic Inference and AI
Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of prob…
Adaptively Solving the Local-Minimum Problem for Deep Neural Networks
This paper aims to overcome a fundamental problem in the theory and application of deep neural networks (DNNs). We propose a method to solve the local minimum problem in training DNNs directly. Our method is based on the…
Information-Theoretic Discrete Diffusion
We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses. Inspired by the I-MMSE identity for the Gaussian setup, we deri…