Evaluating LLM Simulators as Differentially Private Data Generators
LLM-based simulators offer a promising path for generating complex synthetic data where traditional differentially private (DP) methods struggle with high-dimensional user profiles. But can LLMs faithfully reproduce statistical distributions from DP-protected inputs? We evaluate this using PersonaLedger, an agentic financial simulator, seeded with DP synthetic personas derived from real user statistics. We find that PersonaLedger achieves promising fraud detection utility (AUC 0.70 at epsilon=1) but exhibits significant distribution drift due to systematic LLM biases--learned priors overriding input statistics for temporal and demographic features. These failure modes must be addressed before LLM-based methods can handle the richer user representations where they might otherwise excel.
Code (0)
등록된 구현이 없습니다.
Tasks
Fraud DetectionSimilar Papers 제목 키워드 기반
Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model
Differentially private (DP) synthetic data, which closely resembles the original private data while maintaining strong privacy guarantees, has become a key tool for unlocking the value of private data without compromisin…
Image GenerationDifferentially Private Gradient Flow based on the Sliced Wasserstein Distance
Safeguarding privacy in sensitive training data is paramount, particularly in the context of generative modeling. This can be achieved through either differentially private stochastic gradient descent or a differentially…
Private Post-GAN Boosting
Differentially private GANs have proven to be a promising approach for generating realistic synthetic data without compromising the privacy of individuals. Due to the privacy-protective noise introduced in the training, …
Differentially Private Algorithms for Empirical Machine Learning
An important use of private data is to build machine learning classifiers. While there is a burgeoning literature on differentially private classification algorithms, we find that they are not practical in real applicati…
BIG-bench Machine LearningGeneral ClassificationDifferentially Private Policy Evaluation
We present the first differentially private algorithms for reinforcement learning, which apply to the task of evaluating a fixed policy. We establish two approaches for achieving differential privacy, provide a theoretic…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)