Polyharmonic Spline Packages: Composition, Efficient Procedures for Computation and Differentiation
In a previous paper it was shown that a machine learning regression problem can be solved within the framework of random function theory, with the optimal kernel analytically derived from symmetry and indifference principles and coinciding with a polyharmonic spline. However, a direct application of that solution is limited by O(N^3) computational cost and by a breakdown of the original theoretical assumptions when the input space has excessive dimensionality. This paper proposes a cascade architecture built from packages of polyharmonic splines that simultaneously addresses scalability and is theoretically justified for problems with unknown intrinsic low dimensionality. Efficient matrix procedures are presented for forward computation and end-to-end differentiation through the cascade.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Polyharmonic Cascade
This paper presents a deep machine learning architecture, the "polyharmonic cascade" -- a sequence of packages of polyharmonic splines, where each layer is rigorously derived from the theory of random functions and the p…
Large Scale Neural Architecture Search with Polyharmonic Splines
Neural Architecture Search (NAS) is a powerful tool to automatically design deep neural networks for many tasks, including image classification. Due to the significant computational burden of the search phase, most NAS m…
image-classificationImage ClassificationNeural Architecture SearchPolyNeuron: Automatic Neuron Discovery via Learned Polyharmonic Spline Activations
Automated deep neural network architecture design has received a significant amount of recent attention. However, this attention has not been equally shared by one of the fundamental building blocks of a deep neural netw…
Pricing Financial Derivatives using Radial Basis Function generated Finite Differences with Polyharmonic Splines on Smoothly Varying Node Layouts
In this paper, we study the benefits of using polyharmonic splines and node layouts with smoothly varying density for developing robust and efficient radial basis function generated finite difference (RBF-FD) methods for…
Implicit Bias of Gradient Descent for Mean Squared Error Regression with Two-Layer Wide Neural Networks
We investigate gradient descent training of wide neural networks and the corresponding implicit bias in function space. For univariate regression, we show that the solution of training a width-$n$ shallow ReLU network is…
regression