Deep neural networks from the perspective of ergodic theory
The design of deep neural networks remains somewhat of an art rather than precise science. By tentatively adopting ergodic theory considerations on top of viewing the network as the time evolution of a dynamical system, with each layer corresponding to a temporal instance, we show that some rules of thumb, which might otherwise appear mysterious, can be attributed heuristics.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Understanding Oversmoothing in Diffusion-Based GNNs From the Perspective of Operator Semigroup Theory
This paper presents a novel study of the oversmoothing issue in diffusion-based Graph Neural Networks (GNNs). Diverging from extant approaches grounded in random walk analysis or particle systems, we approach this proble…
Node ClassificationThe time interpretation of expected utility theory
Decision theory is the model of individual human behavior employed by neoclassical economics. Built on this model of individual behavior are models of aggregate behavior that feed into models of macroeconomics and inform…
Decision MakingThe Theory and Algorithm of Ergodic Inference
Approximate inference algorithm is one of the fundamental research fields in machine learning. The two dominant theoretical inference frameworks in machine learning are variational inference (VI) and Markov chain Monte C…
BIG-bench Machine LearningVariational InferenceKeynesian chaos revisited: odd period cycles and ergodic properties
In this paper, we study two standard (Keynesian) dynamic macroeconomic models (one is piecewise linear and the other is nonlinear). Our purpose is twofold: (1)~For each model, we give a complete characterisation for the …
Quantum coupled-wave theory of price formation in financial markets: price measurement, dynamics and ergodicity
We explore nature of price formation in financial markets and develop a theory of bid and ask price dynamics in which the two prices form due to quantum-chaotic interaction between buy and sell orders. In this model bid …