Adaptive Bayesian Multivariate Spline Knot Inference with Prior Specifications on Model Complexity
In multivariate spline regression, the number and locations of knots influence the performance and interpretability significantly. However, due to non-differentiability and varying dimensions, there is no desirable frequentist method to make inference on knots. In this article, we propose a fully Bayesian approach for knot inference in multivariate spline regression. The existing Bayesian method often uses BIC to calculate the posterior, but BIC is too liberal and it will heavily overestimate the knot number when the candidate model space is large. We specify a new prior on the knot number to take into account the complexity of the model space and derive an analytic formula in the normal model. In the non-normal cases, we utilize the extended Bayesian information criterion to approximate the posterior density. The samples are simulated in the space with differing dimensions via reversible jump Markov chain Monte Carlo. We apply the proposed method in knot inference and manifold denoising. Experiments demonstrate the splendid capability of the algorithm, especially in function fitting with jumping discontinuity.
Code (1)
Tasks
DenoisingregressionSimilar Papers 제목 키워드 기반
Automatic knot selection in smooth additive models
B-spline regression constitutes a widely used framework for nonparametric modeling. The performance of this methodology depends on specifying the number and placement of changepoints, known as knots, prior to the estimat…
Adaptive spline fitting with particle swarm optimization
In fitting data with a spline, finding the optimal placement of knots can significantly improve the quality of the fit. However, the challenging high-dimensional and non-convex optimization problem associated with comple…
ClusteringModel SelectionThe Upper Bound on Knots in Neural Networks
Neural networks with rectified linear unit activations are essentially multivariate linear splines. As such, one of many ways to measure the "complexity" or "expressivity" of a neural network is to count the number of kn…
Fair Multivariate Adaptive Regression Splines for Ensuring Equity and Transparency
Predictive analytics is widely used in various domains, including education, to inform decision-making and improve outcomes. However, many predictive models are proprietary and inaccessible for evaluation or modification…
Decision MakingFairnessfeature selectionregressionOn the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimators
Kolmogorov-Arnold Networks (KANs) approximate multivariate functions by composing univariate transformations through additive or multiplicative aggregation. We establish convergence guarantees for KANs whose univariate c…