Discrete MDL Predicts in Total Variation
The Minimum Description Length (MDL) principle selects the model that has the shortest code for data plus model. We show that for a countable class of models, MDL predictions are close to the true distribution in a strong sense. The result is completely general. No independence, ergodicity, stationarity, identifiability, or other assumption on the model class need to be made. More formally, we show that for any countable class of models, the distributions selected by MDL (or MAP) asymptotically predict (merge with) the true measure in the class in total variation distance. Implications for non-i.i.d. domains like time-series forecasting, discriminative learning, and reinforcement learning are discussed.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Time SeriesTime Series AnalysisTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Total Variation Classes Beyond 1d: Minimax Rates, and the Limitations of Linear Smoothers
We consider the problem of estimating a function defined over $n$ locations on a $d$-dimensional grid (having all side lengths equal to $n^{1/d}$). When the function is constrained to have discrete total variation bounde…
DenoisingLearnable Total Variation with Lambda Mapping for Low-Dose CT Denoising
While Total Variation (TV) excels in noise reduction and edge preservation, its reliance on a scalar regularization parameter limits adaptivity. In this study, we present a Learnable Total Variation (LTV) framework coupl…
Adaptive Rates for Total Variation Image Denoising
We study the theoretical properties of image denoising via total variation penalized least-squares. We define the total vatiation in terms of the two-dimensional total discrete derivative of the image and show that it gi…
DenoisingImage DenoisingDecorrelated Vectorial Total Variation
This paper proposes a new vectorial total variation prior (VTV) for color images. Different from existing VTVs, our VTV, named the decorrelated vectorial total variation prior (D-VTV), measures the discrete gradients of …
Non-asymptotic approximations for Pearson's chi-square statistic and its application to confidence intervals for strictly convex functions of the probability weights of discrete distributions
In this paper, we develop a non-asymptotic local normal approximation for multinomial probabilities. First, we use it to find non-asymptotic total variation bounds between the measures induced by uniformly jittered multi…