Ensemble Kernel Methods, Implicit Regularization and Determinantal Point Processes
By using the framework of Determinantal Point Processes (DPPs), some theoretical results concerning the interplay between diversity and regularization can be obtained. In this paper we show that sampling subsets with kDPPs results in implicit regularization in the context of ridgeless Kernel Regression. Furthermore, we leverage the common setup of state-of-the-art DPP algorithms to sample multiple small subsets and use them in an ensemble of ridgeless regressions. Our first empirical results indicate that ensemble of ridgeless regressors can be interesting to use for datasets including redundant information.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityPoint ProcessesregressionSimilar Papers 제목 키워드 기반
Determinantal Point Processes Implicitly Regularize Semi-parametric Regression Problems
Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy. Typical examples are spline interpolation in geophysics, or non-linear time series problems…
GeophysicsPoint ProcessesregressionTime Series+1Diversity sampling is an implicit regularization for kernel methods
Kernel methods have achieved very good performance on large scale regression and classification problems, by using the Nystr\"om method and preconditioning techniques. The Nystr\"om approximation -- based on a subset of …
DiversityPoint ProcessesregressionGraphical structure of conditional independencies in determinantal point processes
Determinantal point process have recently been used as models in machine learning and this has raised questions regarding the characterizations of conditional independence. In this paper we investigate characterizations …
BIG-bench Machine LearningPoint ProcessesDuality induced by an embedding structure of determinantal point process
This paper investigates the information geometrical structure of a determinantal point process (DPP). It demonstrates that a DPP is embedded in the exponential family of log-linear models. The extent of deviation from an…
Determinantal consensus clustering
Random restart of a given algorithm produces many partitions to yield a consensus clustering. Ensemble methods such as consensus clustering have been recognized as more robust approaches for data clustering than single c…
ClusteringClustering EnsembleDiversityPoint Processes