Relative Entropy, Probabilistic Inference and AI

2013-03-27 · John E. Shore

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability distributions. In this paper, I will review some basic properties of relative entropy as well as its role in probabilistic inference. I will also mention briefly a few existing and potential applications of relative entropy to so-called artificial intelligence (AI).

📄 PDF Abstract BibTeX arXiv:1304.3423

Code (0)

등록된 구현이 없습니다.

Information Theory with Kernel Methods

2022-02-17 · Francis Bach

We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are…

Variational Inference

Symbolic relative entropy in quantifying nonlinear dynamics of equalities-involved heartbeats

2019-01-28

Symbolic relative entropy, an efficient nonlinear complexity parameter measuring probabilistic divergences of symbolic sequences, is proposed in our nonlinear dynamics analysis of heart rates considering equal states. Eq…

SymCircuit: Bayesian Structure Inference for Tractable Probabilistic Circuits via Entropy-Regularized Reinforcement Learning

2026-03-20 · Y. Sungtaek Ju arxiv

Probabilistic circuit (PC) structure learning is hampered by greedy algorithms that make irreversible, locally optimal decisions. We propose SymCircuit, which replaces greedy search with a learned generative policy train…

Reinforcement Learning

Minimum Relative Entropy Inference for Normal and Monte Carlo Distributions

2020-07-13 · Marcello Colasante, Attilio Meucci

We represent affine sub-manifolds of exponential family distributions as minimum relative entropy sub-manifolds. With such representation we derive analytical formulas for the inference from partial information on expect…

Probabilistic View of Multi-agent Reinforcement Learning: A Unified Approach

2019-09-25 · Shubham Gupta, Ambedkar Dukkipati

Formulating the reinforcement learning (RL) problem in the framework of probabilistic inference not only offers a new perspective about RL, but also yields practical algorithms that are more robust and easier to train. W…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1