Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems
As demonstrated in many areas of real-life applications, neural networks have the capability of dealing with high dimensional data. In the fields of optimal control and dynamical systems, the same capability was studied and verified in many published results in recent years. Towards the goal of revealing the underlying reason why neural networks are capable of solving some high dimensional problems, we develop an algebraic framework and an approximation theory for compositional functions and their neural network approximations. The theoretical foundation is developed in a way so that it supports the error analysis for not only functions as input-output relations, but also numerical algorithms. This capability is critical because it enables the analysis of approximation errors for problems for which analytic solutions are not available, such as differential equations and optimal control. We identify a set of key features of compositional functions and the relationship between the features and the complexity of neural networks. In addition to function approximations, we prove several formulae of error upper bounds for neural networks that approximate the solutions to differential equations, optimization, and optimal control.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Tractable higher-order under-approximating AE extensions for non-linear systems
We consider the problem of under and over-approximating the image of general vector-valued functions over bounded sets, and apply the proposed solution to the estimation of reachable sets of uncertain non-linear discrete…
Compositional Neural Certificates for Networked Dynamical Systems
Developing stable controllers for large-scale networked dynamical systems is crucial but has long been challenging due to two key obstacles: certifiability and scalability. In this paper, we present a general framework t…
Adaptive RKHS Fourier Features for Compositional Gaussian Process Models
Deep Gaussian Processes (DGPs) leverage a compositional structure to model non-stationary processes. DGPs typically rely on local inducing point approximations across intermediate GP layers. Recent advances in DGP infere…
Gaussian ProcessesVariational InferenceCompositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features
Effectively modeling phenomena present in highly nonlinear dynamical systems whilst also accurately quantifying uncertainty is a challenging task, which often requires problem-specific techniques. We present a novel, dom…
Bayesian InferenceGaussian ProcessesregressionTime Series+2Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features
Effectively modeling phenomena present in highly nonlinear dynamical systems whilst also accurately quantifying uncertainty is a challenging task, which often requires problem-specific techniques. We present a novel, dom…
Bayesian InferenceGaussian ProcessesTime SeriesTime Series Analysis+1