Privacy-Preserving Text Classification on BERT Embeddings with Homomorphic Encryption
Embeddings, which compress information in raw text into semantics-preserving low-dimensional vectors, have been widely adopted for their efficacy. However, recent research has shown that embeddings can potentially leak private information about sensitive attributes of the text, and in some cases, can be inverted to recover the original input text. To address these growing privacy challenges, we propose a privatization mechanism for embeddings based on homomorphic encryption, to prevent potential leakage of any piece of information in the process of text classification. In particular, our method performs text classification on the encryption of embeddings from state-of-the-art models like BERT, supported by an efficient GPU implementation of CKKS encryption scheme. We show that our method offers encrypted protection of BERT embeddings, while largely preserving their utility on downstream text classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationGPUPrivacy Preservingtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Natural Language Understanding with Privacy-Preserving BERT
Privacy preservation remains a key challenge in data mining and Natural Language Understanding (NLU). Previous research shows that the input text or even text embeddings can leak private information. This concern motivat…
Language ModellingNatural Language UnderstandingPrivacy PreservingDe-identification of Privacy-related Entities in Job Postings
De-identification is the task of detecting privacy-related entities in text, such as person names, emails and contact data. It has been well-studied within the medical domain. The need for de-identification technology is…
De-identificationMulti-Task LearningPrivacy PreservingMemorization of Named Entities in Fine-tuned BERT Models
Privacy preserving deep learning is an emerging field in machine learning that aims to mitigate the privacy risks in the use of deep neural networks. One such risk is training data extraction from language models that ha…
MemorizationPrivacy PreservingPrivacy Preserving Deep Learningtext-classification+2CAPE: Context-Aware Private Embeddings for Private Language Learning
Deep learning-based language models have achieved state-of-the-art results in a number of applications including sentiment analysis, topic labelling, intent classification and others. Obtaining text representations or em…
intent-classificationIntent ClassificationPrivacy PreservingSentiment AnalysisPrivacy Preserving Topic-wise Sentiment Analysis of the Iran Israel USA Conflict Using Federated Transformer Models
The recent escalation of the Iran Israel USA conflict in 2026 has triggered widespread global discussions across social media platforms. As people increasingly use these platforms for expressing opinions, analyzing publi…
Federated LearningSentiment Analysis