Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks
Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design and training protocol requires careful craftsmanship. Here, we suggest a comprehensive theoretical framework that sheds light on this important problem. Leveraging an equivalence between infinitely over-parameterized neural networks and Gaussian process regression (GPR), we derive an integro-differential equation that governs PINN prediction in the large data-set limit -- the neurally-informed equation. This equation augments the original one by a kernel term reflecting architecture choices and allows quantifying implicit bias induced by the network via a spectral decomposition of the source term in the original differential equation.
Code (0)
등록된 구현이 없습니다.
Tasks
GPRMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scaling Continuous Kernels with Sparse Fourier Domain Learning
We address three key challenges in learning continuous kernel representations: computational efficiency, parameter efficiency, and spectral bias. Continuous kernels have shown significant potential, but their practical a…
Computational EfficiencySparse LearningOptimizing Kernel-Target Alignment for cloud detection in multispectral satellite images
The optimization of Kernel-Target Alignment (TA) has been recently proposed as a way to reduce the number of hardware resources in quantum classifiers. It allows to exchange highly expressive and costly circuits to moder…
Cloud DetectionQuantum Spectral Reasoning: A Non-Neural Architecture for Interpretable Machine Learning
We propose a novel machine learning architecture that departs from conventional neural network paradigms by leveraging quantum spectral methods, specifically Pade approximants and the Lanczos algorithm, for interpretable…
Interpretable Machine LearningAnomaly DetectionSpectral Analysis of Symmetric and Anti-Symmetric Pairwise Kernels
We consider the problem of learning regression functions from pairwise data when there exists prior knowledge that the relation to be learned is symmetric or anti-symmetric. Such prior knowledge is commonly enforced by s…
regressionTopological Neural Tangent Kernel
Graph neural tangent kernels give a principled infinite-width theory for graph neural networks, but inherit a basic limitation of graph models: they see only pairwise structure. Many relational systems contain higher-ord…
Link Prediction