Generic Bounds on the Maximum Deviations in Sequential Prediction: An Information-Theoretic Analysis
In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionSimilar Papers 제목 키워드 기반
Fundamental Limitations in Sequential Prediction and Recursive Algorithms: $\mathcal{L}_{p}$ Bounds via an Entropic Analysis
In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic rel…
Filtering Variational Objectives
When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO t…
Fundamental Limits of Controlled Stochastic Dynamical Systems: An Information-Theoretic Approach
In this paper, we examine the fundamental performance limitations in the control of stochastic dynamical systems; more specifically, we derive generic $\mathcal{L}_p$ bounds that hold for any causal (stabilizing) control…
A Unified Approach to Universal Prediction: Generalized Upper and Lower Bounds
We study sequential prediction of real-valued, arbitrary and unknown sequences under the squared error loss as well as the best parametric predictor out of a large, continuous class of predictors. Inspired by recent resu…
Learning TheoryMixture-of-ExpertsA series of maximum entropy upper bounds of the differential entropy
We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bound…
BIG-bench Machine LearningForm