Sentence-level Privacy for Document Embeddings
User language data can contain highly sensitive personal content. As such, it is imperative to offer users a strong and interpretable privacy guarantee when learning from their data. In this work, we propose SentDP: pure local differential privacy at the sentence level for a single user document. We propose a novel technique, DeepCandidate, that combines concepts from robust statistics and language modeling to produce high-dimensional, general-purpose $\epsilon$-SentDP document embeddings. This guarantees that any single sentence in a document can be substituted with any other sentence while keeping the embedding $\epsilon$-indistinguishable. Our experiments indicate that these private document embeddings are useful for downstream tasks like sentiment analysis and topic classification and even outperform baseline methods with weaker guarantees like word-level Metric DP.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSentenceSentiment AnalysisTopic ClassificationSimilar Papers 제목 키워드 기반
Sentence-level Privacy for Document Embeddings
User language data can contain highly sensitive personal content. As such, it is imperative to offer users a strong and interpretable privacy guarantee when learning from their data. In this work we propose SentDP, pure …
Language ModelingLanguage ModellingSentenceSentiment Analysis+1Hierarchical Document Encoder for Parallel Corpus Mining
We explore using multilingual document embeddings for nearest neighbor mining of parallel data. Three document-level representations are investigated: (i) document embeddings generated by simply averaging multilingual se…
Parallel Corpus MiningSentenceSentence EmbeddingSentence-Embedding+1Neural Attention-Aware Hierarchical Topic Model
Neural topic models (NTMs) apply deep neural networks to topic modelling. Despite their success, NTMs generally ignore two important aspects: (1) only document-level word count information is utilized for the training, w…
modelSentenceTopic ModelsAre the Best Multilingual Document Embeddings simply Based on Sentence Embeddings?
Dense vector representations for textual data are crucial in modern NLP. Word embeddings and sentence embeddings estimated from raw texts are key in achieving state-of-the-art results in various tasks requiring semantic …
SentenceSentence EmbeddingsWord EmbeddingsGranularity is crucial when applying differential privacy to text: An investigation for neural machine translation
Applying differential privacy (DP) by means of the DP-SGD algorithm to protect individual data points during training is becoming increasingly popular in NLP. However, the choice of granularity at which DP is applied is …
Machine TranslationNMTSentence