Variable Selection for Nonparametric Learning with Power Series Kernels
In this paper, we propose a variable selection method for general nonparametric kernel-based estimation. The proposed method consists of two-stage estimation: (1) construct a consistent estimator of the target function, (2) approximate the estimator using a few variables by l1-type penalized estimation. We see that the proposed method can be applied to various kernel nonparametric estimation such as kernel ridge regression, kernel-based density and density-ratio estimation. We prove that the proposed method has the property of the variable selection consistency when the power series kernel is used. This result is regarded as an extension of the variable selection consistency for the non-negative garrote to the kernel-based estimators. Several experiments including simulation studies and real data applications show the effectiveness of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Density Ratio EstimationregressionVariable SelectionSimilar Papers 제목 키워드 기반
Learning Deep Kernels for Non-Parametric Independence Testing
The Hilbert-Schmidt Independence Criterion (HSIC) is a powerful tool for nonparametric detection of dependence between random variables. It crucially depends, however, on the selection of reasonable kernels; commonly-use…
Learning Nonparametric Volterra Kernels with Gaussian Processes
This paper introduces a method for the nonparametric Bayesian learning of nonlinear operators, through the use of the Volterra series with kernels represented using Gaussian processes (GPs), which we term the nonparametr…
Gaussian ProcessesNumerical IntegrationregressionVariational InferenceA Self-Penalizing Objective Function for Scalable Interaction Detection
We tackle the problem of nonparametric variable selection with a focus on discovering interactions between variables. With $p$ variables there are $O(p^s)$ possible order-$s$ interactions making exhaustive search infeasi…
Metric LearningVariable SelectionLearning Stationary Time Series using Gaussian Processes with Nonparametric Kernels
We introduce the Gaussian Process Convolution Model (GPCM), a two-stage nonparametric generative procedure to model stationary signals as the convolution between a continuous-time white-noise process and a continuous-tim…
DenoisingGaussian ProcessesTime SeriesTime Series Analysis+1A Kernel Test for Three-Variable Interactions
We introduce kernel nonparametric tests for Lancaster three-variable interaction and for total independence, using embeddings of signed measures into a reproducing kernel Hilbert space. The resulting test statistics are …