Understanding Learning through the Lens of Dynamical Invariants
This paper proposes a novel perspective on learning, positing it as the pursuit of dynamical invariants -- data combinations that remain constant or exhibit minimal change over time as a system evolves. This concept is underpinned by both informational and physical principles, rooted in the inherent properties of these invariants. Firstly, their stability makes them ideal for memorization and integration into associative networks, forming the basis of our knowledge structures. Secondly, the predictability of these stable invariants makes them valuable sources of usable energy, quantifiable as kTln2 per bit of accurately predicted information. This energy can be harnessed to explore new transformations, rendering learning systems energetically autonomous and increasingly effective. Such systems are driven to continuously seek new data invariants as energy sources. The paper further explores several meta-architectures of autonomous, self-propelled learning agents that utilize predictable information patterns as a source of usable energy.
Code (0)
등록된 구현이 없습니다.
Tasks
MemorizationSimilar Papers 제목 키워드 기반
Preserving linear invariants in ensemble filtering methods
Formulating dynamical models for physical phenomena is essential for understanding the interplay between the different mechanisms and predicting the evolution of physical states. However, a dynamical model alone is often…
Homomorphism Expressivity of Spectral Invariant Graph Neural Networks
Graph spectra are an important class of structural features on graphs that have shown promising results in enhancing Graph Neural Networks (GNNs). Despite their widespread practical use, the theoretical understanding of …
Subgraph CountingCharacterization, Verification and Computation of Robust Controlled Invariants for Monotone Dynamical Systems
In this paper, we consider the problem of computing robust controlled invariants for discrete-time monotone dynamical systems. We consider different classes of monotone systems depending on whether the sets of states, co…
Constraining Chaos: Enforcing dynamical invariants in the training of recurrent neural networks
Drawing on ergodic theory, we introduce a novel training method for machine learning based forecasting methods for chaotic dynamical systems. The training enforces dynamical invariants--such as the Lyapunov exponent spec…
Isomorphism between Differential and Moment Invariants under Affine Transform
The invariant is one of central topics in science, technology and engineering. The differential invariant is essential in understanding or describing some important phenomena or procedures in mathematics, physics, chemis…