Differentially Private Learning Needs Better Features (or Much More Data)
We demonstrate that differentially private machine learning has not yet reached its "AlexNet moment" on many canonical vision tasks: linear models trained on handcrafted features significantly outperform end-to-end deep neural networks for moderate privacy budgets. To exceed the performance of handcrafted features, we show that private learning requires either much more private data, or access to features learned on public data from a similar domain. Our work introduces simple yet strong baselines for differentially private learning that can inform the evaluation of future progress in this area.
Code (2)
Tasks
BIG-bench Machine LearningSimilar Papers 제목 키워드 기반
Representation Transfer for Differentially Private Drug Sensitivity Prediction
Motivation: Human genomic datasets often contain sensitive information that limits use and sharing of the data. In particular, simple anonymisation strategies fail to provide sufficient level of protection for genomic da…
BIG-bench Machine LearningCancer type classificationDimensionality ReductionPrediction+2Classification with Partially Private Features
In this paper, we consider differentially private classification when some features are sensitive, while the rest of the features and the label are not. We adapt the definition of differential privacy naturally to this s…
ClassificationPrivate Selection from Private Candidates
Differentially Private algorithms often need to select the best amongst many candidate options. Classical works on this selection problem require that the candidates' goodness, measured as a real-valued score function, d…
Computational EfficiencyHyperparameter OptimizationDifferentially-Private Decision Trees and Provable Robustness to Data Poisoning
Decision trees are interpretable models that are well-suited to non-linear learning problems. Much work has been done on extending decision tree learning algorithms with differential privacy, a system that guarantees the…
Data PoisoningSelf-Supervised Pretraining for Differentially Private Learning
We demonstrate self-supervised pretraining (SSP) is a scalable solution to deep learning with differential privacy (DP) regardless of the size of available public datasets in image classification. When facing the lack of…
image-classificationImage Classification