Provable wavelet-based neural approximation
In this paper, we develop a wavelet-based theoretical framework for analyzing the universal approximation capabilities of neural networks over a wide range of activation functions. Leveraging wavelet frame theory on the spaces of homogeneous type, we derive sufficient conditions on activation functions to ensure that the associated neural network approximates any functions in the given space, along with an error estimate. These sufficient conditions accommodate a variety of smooth activation functions, including those that exhibit oscillatory behavior. Furthermore, by considering the $L^2$-distance between smooth and non-smooth activation functions, we establish a generalized approximation result that is applicable to non-smooth activations, with the error explicitly controlled by this distance. This provides increased flexibility in the design of network architectures.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Provable approximation properties for deep neural networks
We discuss approximation of functions using deep neural nets. Given a function $f$ on a $d$-dimensional manifold $\Gamma \subset \mathbb{R}^m$, we construct a sparsely-connected depth-4 neural network and bound its error…
Translating Diffusion, Wavelets, and Regularisation into Residual Networks
Convolutional neural networks (CNNs) often perform well, but their stability is poorly understood. To address this problem, we consider the simple prototypical problem of signal denoising, where classical approaches such…
DenoisingDenoising and compression in wavelet domain via projection onto approximation coefficients
We describe a new filtering approach in the wavelet domain for image denoising and compression, based on the projections of details subbands coefficients (resultants of the splitting procedure, typical in wavelet domain)…
DenoisingImage DenoisingAdaptive Online Value Function Approximation with Wavelets
Using function approximation to represent a value function is necessary for continuous and high-dimensional state spaces. Linear function approximation has desirable theoretical guarantees and often requires less compute…
AcrobotFrom Spectrum Wavelet to Vertex Propagation: Graph Convolutional Networks Based on Taylor Approximation
Graph convolutional networks (GCN) have been recently utilized to extract the underlying structures of datasets with some labeled data and high-dimensional features. Existing GCNs mostly rely on a first-order Chebyshev a…
Node Classification