Geometric separation and constructive universal approximation with two hidden layers
We give a geometric construction of neural networks that separate disjoint compact subsets of $\Bbb R^n$, and use it to obtain a constructive universal approximation theorem. Specifically, we show that networks with two hidden layers and either a sigmoidal activation (i.e., strictly monotone bounded continuous) or the ReLU activation can approximate any real-valued continuous function on an arbitrary compact set $K\subset\Bbb R^n$ to any prescribed accuracy in the uniform norm. For finite $K$, the construction simplifies and yields a sharp depth-2 (single hidden layer) approximation result.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Two-hidden-layer Feedforward Neural Networks are Universal Approximators: A Constructive Approach
It is well known that Artificial Neural Networks are universal approximators. The classical result proves that, given a continuous function on a compact set on an n-dimensional space, then there exists a one-hidden-layer…
Universal Approximation Theorems of Fully Connected Binarized Neural Networks
Neural networks (NNs) are known for their high predictive accuracy in complex learning problems. Beside practical advantages, NNs also indicate favourable theoretical properties such as universal approximation (UA) theor…
LightGCNet: A Lightweight Geometric Constructive Neural Network for Data-Driven Soft sensors
Data-driven soft sensors provide a potentially cost-effective and more accurate modeling approach to measure difficult-to-measure indices in industrial processes compared to mechanistic approaches. Artificial intelligenc…
On the approximation by single hidden layer feedforward neural networks with fixed weights
Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) wi…
Simplicial Neural Networks:First Steps and Future Applications
It is well-known that artificial neural networks are universal approximators. The Universal Approximation Theorem proves that, given a continuous function on a compact set embedded in an $n$-dimensional space, there exis…