A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness
The aim of dataset distillation is to encode the rich features of an original dataset into a tiny dataset. It is a promising approach to accelerate neural network training and related studies. Different approaches have been proposed to improve the informativeness and generalization performance of distilled images. However, no work has comprehensively analyzed this technique from a security perspective and there is a lack of systematic understanding of potential risks. In this work, we conduct extensive experiments to evaluate current state-of-the-art dataset distillation methods. We successfully use membership inference attacks to show that privacy risks still remain. Our work also demonstrates that dataset distillation can cause varying degrees of impact on model robustness and amplify model unfairness across classes when making predictions. This work offers a large-scale benchmarking framework for dataset distillation evaluation.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDataset DistillationFairnessInformativenessSimilar Papers 제목 키워드 기반
On Membership Inference Attacks in Knowledge Distillation
Nowadays, Large Language Models (LLMs) are trained on huge datasets, some including sensitive information. This poses a serious privacy concern because privacy attacks such as Membership Inference Attacks (MIAs) may dete…
Knowledge DistillationPrivacy PreservingUnveiling Client Privacy Leakage from Public Dataset Usage in Federated Distillation
Federated Distillation (FD) has emerged as a popular federated training framework, enabling clients to collaboratively train models without sharing private data. Public Dataset-Assisted Federated Distillation (PDA-FD), w…
Federated LearningInference AttackStudents Parrot Their Teachers: Membership Inference on Model Distillation
Model distillation is frequently proposed as a technique to reduce the privacy leakage of machine learning. These empirical privacy defenses rely on the intuition that distilled ``student'' models protect the privacy of …
Knowledge DistillationPrivacy Evaluation Benchmarks for NLP Models
By inducing privacy attacks on NLP models, attackers can obtain sensitive information such as training data and model parameters, etc. Although researchers have studied, in-depth, several kinds of attacks in NLP models, …
Knowledge DistillationSwing Distillation: A Privacy-Preserving Knowledge Distillation Framework
Knowledge distillation (KD) has been widely used for model compression and knowledge transfer. Typically, a big teacher model trained on sufficient data transfers knowledge to a small student model. However, despite the …
Knowledge DistillationModel CompressionPrivacy PreservingTransfer Learning