Interpolation, Approximation and Controllability of Deep Neural Networks
We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that arise from supervised learning, namely universal interpolation - the ability to match arbitrary input and target training samples - and the closely related notion of universal approximation - the ability to approximate input-target functional relationships via flow maps. Under the assumption of affine invariance of the control family, we give a characterisation of universal interpolation, showing that it holds for essentially any architecture with non-linearity. Furthermore, we elucidate the relationship between universal interpolation and universal approximation in the context of general control systems, showing that the two properties cannot be deduced from each other. At the same time, we identify conditions on the control family and the target function that ensures the equivalence of the two notions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Interplay between depth and width for interpolation in neural ODEs
Neural ordinary differential equations (neural ODEs) have emerged as a natural tool for supervised learning from a control perspective, yet a complete understanding of their optimal architecture remains elusive. In this …
Distributed filtered hyperinterpolation for noisy data on the sphere
Problems in astrophysics, space weather research and geophysics usually need to analyze noisy big data on the sphere. This paper develops distributed filtered hyperinterpolation for noisy data on the sphere, which assign…
GeophysicsModel SelectionA New Interpolation Approach and Corresponding Instance-Based Learning
Starting from finding approximate value of a function, introduces the measure of approximation-degree between two numerical values, proposes the concepts of "strict approximation" and "strict approximation region", then,…
Cloud ComputingInductive LearningHNS: An Efficient Hermite Neural Solver for Solving Time-Fractional Partial Differential Equations
Neural network solvers represent an innovative and promising approach for tackling time-fractional partial differential equations by utilizing deep learning techniques. L1 interpolation approximation serves as the standa…
Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theory
In this paper, we explain the universal approximation capabilities of deep residual neural networks through geometric nonlinear control. Inspired by recent work establishing links between residual networks and control sy…