Deep Neural Network Approximation of Invariant Functions through Dynamical Systems
We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions includes the much studied translation-invariant ones involving image tasks, but also encompasses many permutation-invariant functions that finds emerging applications in science and engineering. We prove sufficient conditions for universal approximation of these functions by a controlled equivariant dynamical system, which can be viewed as a general abstraction of deep residual networks with symmetry constraints. These results not only imply the universal approximation for a variety of commonly employed neural network architectures for symmetric function approximation, but also guide the design of architectures with approximation guarantees for applications involving new symmetry requirements.
Code (0)
등록된 구현이 없습니다.
Tasks
TranslationSimilar Papers 제목 키워드 기반
Polynomial Lyapunov Functions and Invariant Sets from a New Hierarchy of Quadratic Lyapunov Functions for LTV Systems
We introduce a new class of quadratic functions based on a hierarchy of linear time-varying (LTV) dynamical systems. These quadratic functions in the higher order space can be also seen as a non-homogeneous polynomial Ly…
Learning dynamically inspired invariant subspaces for Koopman and transfer operator approximation
Transfer and Koopman operator methods offer a framework for representing complex, nonlinear dynamical systems via linear transformations, enabling for a deeper understanding of the underlying dynamics. The spectrum of th…
Metric Entropy Limits on Recurrent Neural Network Learning of Linear Dynamical Systems
One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer…
On the Universal Approximation Property of Deep Fully Convolutional Neural Networks
We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective. We prove that deep residual fully convolutional networks and their conti…
Replacing K-infinity Function with Leaky ReLU in Barrier Function Design: A Union of Invariant Sets Approach for ReLU-Based Dynamical Systems
In this paper, a systematic framework is presented for determining piecewise affine PWA barrier functions and their corresponding invariant sets for dynamical systems identified via Rectified Linear Unit (ReLU) neural ne…