Characterization of the probability and information entropy of a process with an exponentially increasing sample space and its application to the Broad Money Supply
There is a random variable (X) with a determined outcome (i.e., X = x0), p(x0) = 1. Consider x0 to have a discrete uniform distribution over the integer interval [1, s], where the size of the sample space (s) = 1, in the initial state, such that p(x0) = 1. What is the probability of x0 and the associated information entropy (H), as s increases exponentially? If the sample space expansion occurs at an exponential rate (rate constant = lambda) with time (t) and applying time scaling, such that T = lambda x t, gives: p(x0|T)=exp(-T) and H(T)=T. The characterization has also been extended to include exponential expansion by means of simultaneous, independent processes, as well as the more general multi-exponential case. The methodology was applied to the expansion of the broad money supply of US$ over the period 2001-2019, as a real-world example. At any given time, the information entropy is related to the rate at which the sample space is expanding. In the context of the expansion of the broad money supply, the information entropy could be considered to be related to the "velocity" of the expansion of the money supply.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Characterization of the probability and information entropy of a process with an increasing sample space by different functional forms of expansion, with an application to hyperinflation
There is a random variable (X) with a determined outcome (i.e., X = x0), p(x0) = 1. Consider x0 to have a discrete uniform distribution over the integer interval [1, s], where the size of the sample space (s) = 1, in the…
Optimal Change-Point Detection with Training Sequences in the Large and Moderate Deviations Regimes
This paper investigates a novel offline change-point detection problem from an information-theoretic perspective. In contrast to most related works, we assume that the knowledge of the underlying pre- and post-change dis…
Change Point DetectionA Note on the Estimation of Von Neumann and Relative Entropy via Quantum State Observers
An essential quantity in quantum information theory is the von Neumann entropy which depends entirely on the quantum density operator. Once known, the density operator reveals the statistics of observables in a quantum p…
Metric-Entropy Limits on the Approximation of Nonlinear Dynamical Systems
This paper is concerned with fundamental limits on the approximation of nonlinear dynamical systems. Specifically, we show that recurrent neural networks (RNNs) can approximate nonlinear systems -- that satisfy a Lipschi…
Entropy Production in Machine Learning Under Fokker-Planck Probability Flow
Machine learning models deployed in nonstationary environments inevitably experience performance degradation due to data drift. While numerous drift detection heuristics exist, most lack a dynamical interpretation and pr…