An Application of the Holonomic Gradient Method to the Neural Tangent Kernel
A holonomic system of linear partial differential equations is, roughly speaking, a system whose solution space is finite dimensional. A distribution that is a solution of a holonomic system is called a holonomic distribution. We give methods to numerically evaluate dual activations of holonomic activator distributions for neural tangent kernels. These methods are based on computer algebra algorithms for rings of differential operators.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning the Rolling Penny Dynamics
We consider learning the dynamics of a typical nonholonomic system -- the rolling penny. A nonholonomic system is a system subject to nonholonomic constraints. Unlike a holonomic constraints, a nonholonomic constraint do…
Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels
We study the relative power of learning with gradient descent on differentiable models, such as neural networks, versus using the corresponding tangent kernels. We show that under certain conditions, gradient descent ach…
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks
A recent breakthrough in deep learning theory shows that the training of over-parameterized deep neural networks can be characterized by a kernel function called \textit{neural tangent kernel} (NTK). However, it is known…
Learning TheoryVocal Bursts Valence PredictionNonlocal Neural Tangent Kernels via Parameter-Space Interactions
The Neural Tangent Kernel (NTK) framework has provided deep insights into the training dynamics of neural networks under gradient flow. However, it relies on the assumption that the network is differentiable with respect…
Deciphering Neural Reparameterized Full-Waveform Inversion with Neural Sensitivity Kernel and Wave Tangent Kernel
Full-waveform inversion (FWI) estimates unknown parameters in the wave equation from limited boundary measurements. Recent advances in neural reparameterized FWI (NeurFWI) demonstrate that representing the parameters usi…