Product Interaction: An Algebraic Formalism for Deep Learning Architectures
In this paper, we introduce product interactions, an algebraic formalism in which neural network layers are constructed from compositions of a multiplication operator defined over suitable algebras. Product interactions provide a principled way to generate and organize algebraic expressions by increasing interaction order. Our central observation is that algebraic expressions in modern neural networks admit a unified construction in terms of linear, quadratic, and higher-order product interactions. Convolutional and equivariant networks arise as symmetry-constrained linear product interactions, while attention and Mamba correspond to higher-order product interactions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Information algebras in the theory of imprecise probabilities
In this paper, we show that coherent sets of gambles and coherent lower and upper previsions can be embedded into the algebraic structure of information algebra. This leads firstly, to a new perspective of the algebraic …
CliffordNet: All You Need is Geometric Algebra
Modern computer vision architectures, from CNNs to Transformers, predominantly rely on the stacking of heuristic modules: spatial mixers (Attention/Conv) followed by channel mixers (FFNs). In this work, we challenge this…
Ternary Gamma Semirings as a Novel Algebraic Framework for Learnable Symbolic Reasoning
Binary semirings such as the tropical, log, and probability semirings form a core algebraic tool in classical and modern neural inference systems, supporting tasks like Viterbi decoding, dynamic programming, and probabil…
Knowledge GraphsLinguistics and some aspects of its underlying dynamics
In recent years, central components of a new approach to linguistics, the Minimalist Program (MP) have come closer to physics. Features of the Minimalist Program, such as the unconstrained nature of recursive Merge, the …
SentenceCoalgebras for categorical deep learning: Representability and universal approximation
Categorical deep learning (CDL) has recently emerged as a framework that leverages category theory to unify diverse neural architectures. While geometric deep learning (GDL) is grounded in the specific context of invaria…