Querying Kernel Methods Suffices for Reconstructing their Training Data
Over-parameterized models have raised concerns about their potential to memorize training data, even when achieving strong generalization. The privacy implications of such memorization are generally unclear, particularly in scenarios where only model outputs are accessible. We study this question in the context of kernel methods, and demonstrate both empirically and theoretically that querying kernel models at various points suffices to reconstruct their training data, even without access to model parameters. Our results hold for a range of kernel methods, including kernel regression, support vector machines, and kernel density estimation. Our hope is that this work can illuminate potential privacy concerns for such models.
Code (1)
Tasks
Density EstimationMemorizationSimilar Papers 제목 키워드 기반
Reconstruction of Graph Signals on Complex Manifolds with Kernel Methods
Graph signals are widely used to describe vertex attributes or features in graph-structured data, with applications spanning the internet, social media, transportation, sensor networks, and biomedicine. Graph signal proc…
Streaming Kernel PCA with \tilde{O}(\sqrt{n}) Random Features
We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/\epsilo…
Streaming Kernel PCA with $\tilde{O}(\sqrt{n})$ Random Features
We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/\epsilo…
Neural Fields as Learnable Kernels for 3D Reconstruction
We present Neural Kernel Fields: a novel method for reconstructing implicit 3D shapes based on a learned kernel ridge regression. Our technique achieves state-of-the-art results when reconstructing 3D objects and large s…
3D ReconstructionInductive BiasregressionActive Learning Methods based on Statistical Leverage Scores
In many real-world machine learning applications, unlabeled data are abundant whereas class labels are expensive and scarce. An active learner aims to obtain a model of high accuracy with as few labeled instances as poss…
Active Learning