Differential Privacy Mechanisms in Neural Tangent Kernel Regression
Training data privacy is a fundamental problem in modern Artificial Intelligence (AI) applications, such as face recognition, recommendation systems, language generation, and many others, as it may contain sensitive user information related to legal issues. To fundamentally understand how privacy mechanisms work in AI applications, we study differential privacy (DP) in the Neural Tangent Kernel (NTK) regression setting, where DP is one of the most powerful tools for measuring privacy under statistical learning, and NTK is one of the most popular analysis frameworks for studying the learning mechanisms of deep neural networks. In our work, we can show provable guarantees for both differential privacy and test accuracy of our NTK regression. Furthermore, we conduct experiments on the basic image classification dataset CIFAR10 to demonstrate that NTK regression can preserve good accuracy under a modest privacy budget, supporting the validity of our analysis. To our knowledge, this is the first work to provide a DP guarantee for NTK regression.
Code (0)
등록된 구현이 없습니다.
Tasks
Face Recognitionimage-classificationImage ClassificationRecommendation SystemsregressionText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Tangent differential privacy
Differential privacy is a framework for protecting the identity of individual data points in the decision-making process. In this note, we propose a new form of differential privacy called tangent differential privacy. C…
Decision MakingDifferentially Private Neural Tangent Kernels for Privacy-Preserving Data Generation
Maximum mean discrepancy (MMD) is a particularly useful distance metric for differentially private data generation: when used with finite-dimensional features it allows us to summarize and privatize the data distribution…
Privacy PreservingDeconvoluting Kernel Density Estimation and Regression for Locally Differentially Private Data
Local differential privacy has become the gold-standard of privacy literature for gathering or releasing sensitive individual data points in a privacy-preserving manner. However, locally differential data can twist the p…
Density EstimationPrivacy PreservingregressionAuditing Differential Privacy in High Dimensions with the Kernel Quantum Rényi Divergence
Differential privacy (DP) is the de facto standard for private data release and private machine learning. Auditing black-box DP algorithms and mechanisms to certify whether they satisfy a certain DP guarantee is challeng…
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
The evolution of a deep neural network trained by the gradient descent can be described by its neural tangent kernel (NTK) as introduced in [20], where it was proven that in the infinite width limit the NTK converges to …
regression