On the relationship between multivariate splines and infinitely-wide neural networks
We consider multivariate splines and show that they have a random feature expansion as infinitely wide neural networks with one-hidden layer and a homogeneous activation function which is the power of the rectified linear unit. We show that the associated function space is a Sobolev space on a Euclidean ball, with an explicit bound on the norms of derivatives. This link provides a new random feature expansion for multivariate splines that allow efficient algorithms. This random feature expansion is numerically better behaved than usual random Fourier features, both in theory and practice. In particular, in dimension one, we compare the associated leverage scores to compare the two random expansions and show a better scaling for the neural network expansion.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks
We present Neural Splines, a technique for 3D surface reconstruction that is based on random feature kernels arising from infinitely-wide shallow ReLU networks. Our method achieves state-of-the-art results, outperforming…
Surface ReconstructionScalable Bayesian Network Structure Learning with Splines
The graph structure of a Bayesian network (BN) can be learned from data using the well-known score-and-search approach. Previous work has shown that incorporating structured representations of the conditional probability…
feature selectionregressionSMART: A Flexible Approach to Regression using Spline-Based Multivariate Adaptive Regression Trees
Decision trees are powerful for predictive modeling but often suffer from high variance when modeling continuous relationships. While algorithms like Multivariate Adaptive Regression Splines (MARS) excel at capturing suc…
regressionFair Multivariate Adaptive Regression Splines for Ensuring Equity and Transparency
Predictive analytics is widely used in various domains, including education, to inform decision-making and improve outcomes. However, many predictive models are proprietary and inaccessible for evaluation or modification…
Decision MakingFairnessfeature selectionregressionIntrusion Detection Systems Using Adaptive Regression Splines
Past few years have witnessed a growing recognition of intelligent techniques for the construction of efficient and reliable intrusion detection systems. Due to increasing incidents of cyber attacks, building effective i…
Intrusion Detectionregression