Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning
We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit problem. We contrast our analysis with computation-theoretic and statistical inference approaches to understanding their complexity. The result is an alternative view of the relationship between predictability, complexity, and learning that highlights the distinct ways in which informational and correlational divergences arise in complex ergodic and nonergodic processes. We draw out consequences for the resource divergences that delineate the structural hierarchy of ergodic processes and for processes that are themselves hierarchical.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis
We present Infinity, a Bitwise Visual AutoRegressive Modeling capable of generating high-resolution, photorealistic images following language instruction. Infinity redefines visual autoregressive model under a bitwise to…
Image GenerationInfinityGAN: Towards Infinite-Pixel Image Synthesis
We present a novel framework, InfinityGAN, for arbitrary-sized image generation. The task is associated with several key challenges. First, scaling existing models to an arbitrarily large image size is resource-constrain…
Image GenerationScene GenerationRobust Scaling in Human Brain Dynamics Despite Latent Variables and Limited Sampling Distortions
The idea that information-processing systems operate near criticality to enhance computational performance is supported by scaling signatures in brain activity. However, external signals raise the question of whether thi…
On Bridging the Gap between Mean Field and Finite Width in Deep Random Neural Networks with Batch Normalization
Mean field theory is widely used in the theoretical studies of neural networks. In this paper, we analyze the role of depth in the concentration of mean-field predictions, specifically for deep multilayer perceptron (MLP…
PointInfinity: Resolution-Invariant Point Diffusion Models
We present PointInfinity, an efficient family of point cloud diffusion models. Our core idea is to use a transformer-based architecture with a fixed-size, resolution-invariant latent representation. This enables efficien…