Driving Context into Text-to-Text Privatization
\textit{Metric Differential Privacy} enables text-to-text privatization by adding calibrated noise to the vector of a word derived from an embedding space and projecting this noisy vector back to a discrete vocabulary using a nearest neighbor search. Since words are substituted without context, this mechanism is expected to fall short at finding substitutes for words with ambiguous meanings, such as \textit{'bank'}. To account for these ambiguous words, we leverage a sense embedding and incorporate a sense disambiguation step prior to noise injection. We encompass our modification to the privatization mechanism with an estimation of privacy and utility. For word sense disambiguation on the \textit{Words in Context} dataset, we demonstrate a substantial increase in classification accuracy by $6.05\%$.
Code (0)
등록된 구현이 없습니다.
Tasks
Word Sense DisambiguationSimilar Papers 제목 키워드 기반
Guiding Text-to-Text Privatization by Syntax
Metric Differential Privacy is a generalization of differential privacy tailored to address the unique challenges of text-to-text privatization. By adding noise to the representation of words in the geometric space of em…
Disentangling the Linguistic Competence of Privacy-Preserving BERT
Differential Privacy (DP) has been tailored to address the unique challenges of text-to-text privatization. However, text-to-text privatization is known for degrading the performance of language models when trained on pe…
Privacy PreservingCharacterizing Stereotypical Bias from Privacy-preserving Pre-Training
Differential Privacy (DP) can be applied to raw text by exploiting the spatial arrangement of words in an embedding space. We investigate the implications of such text privatization on Language Models (LMs) and their ten…
Language ModelingLanguage ModellingPrivacy PreservingWhen does privatization spur entrepreneurial performance? The moderating effect of institutional quality in an emerging market
We explore how institutional quality moderates the effectiveness of privatization on entrepreneurs sales performance. To do this, we blend agency theory and entrepreneurial cognition theory with insights from institution…
Investigating User Perspectives on Differentially Private Text Privatization
Recent literature has seen a considerable uptick in $\textit{Differentially Private Natural Language Processing}$ (DP NLP). This includes DP text privatization, where potentially sensitive input texts are transformed und…