TabPATE: Differentially Private Tabular In-Context Learning Without Public Data
Tabular foundation models enable accurate in-context learning (ICL) from small labeled datasets, but the private records placed in context can leak through model predictions. We first show that even basic membership inference attacks succeed against tabular ICL, motivating formal privacy protection. We then introduce TabPATE, a differentially private PATE-style defense for tabular ICL that does not require public in-distribution data. TabPATE partitions the private context across teacher models, privately aggregates their labels on synthetic tabular queries, and releases the resulting labeled queries as a student context. Because tabular features are bounded and relatively low-dimensional, useful queries can be generated from feature ranges alone or from lightly privatized marginals. Across tabular benchmarks, TabPATE preserves competitive utility while reducing membership inference to near-random success, providing a practical path to private tabular ICL without public data.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators
Generating tabular data under differential privacy (DP) protection ensures theoretical privacy guarantees but poses challenges for training machine learning models, primarily due to the need to capture complex structures…
Tabular Data GenerationDifferentially Private Synthetic Data: Applied Evaluations and Enhancements
Machine learning practitioners frequently seek to leverage the most informative available data, without violating the data owner's privacy, when building predictive models. Differentially private data synthesis protects …
BIG-bench Machine LearningDP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation
The generation of synthetic tabular data that preserves differential privacy is a problem of growing importance. While traditional marginal-based methods have achieved impressive results, recent work has shown that deep …
Deep LearningTabular Data GenerationDifferentially Private Tabular Data Synthesis using Large Language Models
Synthetic tabular data generation with differential privacy is a crucial problem to enable data sharing with formal privacy. Despite a rich history of methodological research and development, developing differentially pr…
FairnessTabular Data GenerationDifferentially Private Synthetic High-dimensional Tabular Stream
While differentially private synthetic data generation has been explored extensively in the literature, how to update this data in the future if the underlying private data changes is much less understood. We propose an …
Synthetic Data Generation