Algebraic Invariants of Lightning Self-Attention
We study the polynomial coefficients of lightning self-attention as coordinates of an algebraic variety. We identify linear and nonlinear families of algebraic invariants, including Chow-type, low-rank, Veronese-type, and Sylvester resultant-based constraints.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Geometry of Lightning Self-Attention: Identifiability and Dimension
We consider function spaces defined by self-attention networks without normalization, and theoretically analyze their geometry. Since these networks are polynomial, we rely on tools from algebraic geometry. In particular…
Deep AttentionRobustness Verification of Polynomial Neural Networks
We study robustness verification of neural networks via metric algebraic geometry. For polynomial neural networks, certifying a robustness radius amounts to computing the distance to the algebraic decision boundary. We u…
On the Invariants of Softmax Attention
Softmax attention maps every query--key interaction into a probability distribution, but the underlying structure remains largely unexplored. We define the \emph{energy field}, the row-centered attention logit, and show …
Computing algebraic degrees of phylogenetic varieties
A phylogenetic variety is an algebraic variety parameterized by a statistical model of the evolution of biological sequences along a tree. Understanding this variety is an important problem in the area of algebraic stati…
Phylogenetic invariants: straightforward from the general Markov to equivariant models
In the last decade, some algebraic tools have been successfully applied to phylogenetic reconstruction. These tools are mainly based on the knowledge of equations describing algebraic varieties associated to phylogenetic…