Algebraic Dynamical Systems in Machine Learning
We introduce an algebraic analogue of dynamical systems, based on term rewriting. We show that a recursive function applied to the output of an iterated rewriting system defines a formal class of models into which all the main architectures for dynamic machine learning models (including recurrent neural networks, graph neural networks, and diffusion models) can be embedded. Considered in category theory, we also show that these algebraic models are a natural language for describing the compositionality of dynamic models. Furthermore, we propose that these models provide a template for the generalisation of the above dynamic models to learning problems on structured or non-numerical data, including 'hybrid symbolic-numeric' models.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Computational Dynamical Systems
We study the computational complexity theory of smooth, finite-dimensional dynamical systems. Building off of previous work, we give definitions for what it means for a smooth dynamical system to simulate a Turing machin…
The $k$-Compound of a Difference-Algebraic System
The multiplicative and additive compounds of a matrix have important applications in geometry, linear algebra, and the analysis of dynamical systems. In particular, the $k$-compounds allow to build a $k$-compound dynamic…
Learnable & Interpretable Model Combination in Dynamical Systems Modeling
During modeling of dynamical systems, often two or more model architectures are combined to obtain a more powerful or efficient model regarding a specific application area. This covers the combination of multiple machine…
modelPhysical SimulationsCausal discovery in deterministic discrete LTI-DAE systems
Discovering pure causes or driver variables in deterministic LTI systems is of vital importance in the data-driven reconstruction of causal networks. A recent work by Kathari and Tangirala, proposed in 2022, formulated t…
Causal DiscoveryAlgebraic Systems Biology: A Case Study for the Wnt Pathway
Steady state analysis of dynamical systems for biological networks give rise to algebraic varieties in high-dimensional spaces whose study is of interest in their own right. We demonstrate this for the shuttle model of t…