Understanding Kernel Ridge Regression: Common behaviors from simple functions to density functionals
Accurate approximations to density functionals have recently been obtained via machine learning (ML). By applying ML to a simple function of one variable without any random sampling, we extract the qualitative dependence of errors on hyperparameters. We find universal features of the behavior in extreme limits, including both very small and very large length scales, and the noise-free limit. We show how such features arise in ML models of density functionals.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningregressionSimilar Papers 제목 키워드 기반
Asymptotic Optimism of Random-Design Linear and Kernel Regression Models
We derived the closed-form asymptotic optimism of linear regression models under random designs, and generalizes it to kernel ridge regression. Using scaled asymptotic optimism as a generic predictive model complexity me…
regressionUniform convergence for Gaussian kernel ridge regression
This paper establishes the first polynomial convergence rates for Gaussian kernel ridge regression (KRR) with a fixed hyperparameter in both the uniform and the $L^{2}$-norm. The uniform convergence result closes a gap i…
A Comparative Study of Pairwise Learning Methods based on Kernel Ridge Regression
Many machine learning problems can be formulated as predicting labels for a pair of objects. Problems of that kind are often referred to as pairwise learning, dyadic prediction or network inference problems. During the l…
BIG-bench Machine LearningregressionZero-Shot LearningLarge Dimensional Kernel Ridge Regression: Extending to Product Kernels
Recent studies have reported $\textit{saturation effects}$ and $\textit{multiple descent behavior}$ in large dimensional kernel ridge regression (KRR). However, these findings are predominantly derived under restrictive …
How rotational invariance of common kernels prevents generalization in high dimensions
Kernel ridge regression is well-known to achieve minimax optimal rates in low-dimensional settings. However, its behavior in high dimensions is much less understood. Recent work establishes consistency for kernel regress…
regressionVocal Bursts Intensity Prediction