Provable approximation properties for deep neural networks
We discuss approximation of functions using deep neural nets. Given a function $f$ on a $d$-dimensional manifold $\Gamma \subset \mathbb{R}^m$, we construct a sparsely-connected depth-4 neural network and bound its error in approximating $f$. The size of the network depends on dimension and curvature of the manifold $\Gamma$, the complexity of $f$, in terms of its wavelet description, and only weakly on the ambient dimension $m$. Essentially, our network computes wavelet functions, which are computed from Rectified Linear Units (ReLU)
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Online Model Order Reduction of Linear Systems via $(γ,δ)$-Similarity
Model order reduction aims to determine a low-order approximation of high-order models with least possible approximation errors. For application to physical systems, it is crucial that the reduced order model (ROM) is ro…
Provable Smoothness Guarantees for Black-Box Variational Inference
Black-box variational inference tries to approximate a complex target distribution though a gradient-based optimization of the parameters of a simpler distribution. Provable convergence guarantees require structural prop…
Variational InferenceOn the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shift
Policy gradient methods are among the most effective methods in challenging reinforcement learning problems with large state and/or action spaces. However, little is known about even their most basic theoretical converge…
Policy Gradient MethodsReinforcement LearningInfinite-dimensional reservoir computing
Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems that extends the so-called generalized Barron functionals to a dynamic context. This new class is cha…
Generalization BoundsProvable Preimage Under-Approximation for Neural Networks (Full Version)
Neural network verification mainly focuses on local robustness properties, which can be checked by bounding the image (set of outputs) of a given input set. However, often it is important to know whether a given property…