EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have emerged as powerful engines for generating it. However, existing private text generation methods are severely inefficient: they are data-intensive, computationally slow, and often require large private corpora or batch sizes to achieve usable quality. We introduce EPSVec, a differentially-private lightweight alternative that steers LLM generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors. EPSVec extracts and sanitizes steering vectors just once and then performs standard decoding. This decouples the privacy budget from generation, enabling arbitrarily many synthetic samples without additional privacy cost and yielding strong fidelity even in low-data regimes. Furthermore, we enhance our method by utilizing pretrained (base) models and introducing fixed-shot prompting to boost generation diversity and fidelity. Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility, particularly in low-data regimes, while significantly reducing computational overhead.
Code (0)
등록된 구현이 없습니다.
Tasks
Synthetic Data GenerationText GenerationSimilar Papers 제목 키워드 기반
Differentially Private Synthetic High-dimensional Tabular Stream
While differentially private synthetic data generation has been explored extensively in the literature, how to update this data in the future if the underlying private data changes is much less understood. We propose an …
Synthetic Data GenerationPrivate Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
Large language models (LLMs) have emerged as a powerful tool for synthetic data generation. A particularly important use case is producing synthetic replicas of private text, which requires carefully balancing privacy an…
Synthetic Data GenerationStruct-Bench: A Benchmark for Differentially Private Structured Text Generation
Differentially private (DP) synthetic data generation is a promising technique for utilizing private datasets that otherwise cannot be exposed for model training or other analytics. While much research literature has foc…
Synthetic Data GenerationSynthetic Data EvaluationText GenerationLeveraging Vertical Public-Private Split for Improved Synthetic Data Generation
Differentially Private Synthetic Data Generation (DP-SDG) is a key enabler of private and secure tabular-data sharing, producing artificial data that carries through the underlying statistical properties of the input dat…
Synthetic Data GenerationPrivate Synthetic Text Generation with Diffusion Models
How capable are diffusion models of generating synthetics texts? Recent research shows their strengths, with performance reaching that of auto-regressive LLMs. But are they also good in generating synthetic data if the t…
Image GenerationText Generation