Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment

2024-07-08 · Qizhang Feng, Siva Rajesh Kasa, Hyokun Yun, Choon Hui Teo, Sravan Babu Bodapati

Large Language Models (LLMs) have seen widespread adoption due to their remarkable natural language capabilities. However, when deploying them in real-world settings, it is important to align LLMs to generate texts according to acceptable human standards. Methods such as Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO) have made significant progress in refining LLMs using human preference data. However, the privacy concerns inherent in utilizing such preference data have yet to be adequately studied. In this paper, we investigate the vulnerability of LLMs aligned using human preference datasets to membership inference attacks (MIAs), highlighting the shortcomings of previous MIA approaches with respect to preference data. Our study has two main contributions: first, we introduce a novel reference-based attack framework specifically for analyzing preference data called PREMIA (\uline{Pre}ference data \uline{MIA}); second, we provide empirical evidence that DPO models are more vulnerable to MIA compared to PPO models. Our findings highlight gaps in current privacy-preserving practices for LLM alignment.

📄 PDF Abstract BibTeX arXiv:2407.06443

Code (0)

등록된 구현이 없습니다.

Tasks

Inference AttackMembership Inference AttackPrivacy Preserving

Methods 이 논문이 사용한 방법론

DPO 설명 없음

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

On Primes, Log-Loss Scores and (No) Privacy

2020-09-17 · Abhinav Aggarwal, Zekun Xu, Oluwaseyi Feyisetan, Nathanael Teissier

Membership Inference Attacks exploit the vulnerabilities of exposing models trained on customer data to queries by an adversary. In a recently proposed implementation of an auditing tool for measuring privacy leakage fro…

Active Membership Inference Attack under Local Differential Privacy in Federated Learning

2023-02-24 · Truc Nguyen, Phung Lai, Khang Tran, NhatHai Phan 외

Federated learning (FL) was originally regarded as a framework for collaborative learning among clients with data privacy protection through a coordinating server. In this paper, we propose a new active membership infere…

Federated Learning Inference Attack Membership Inference Attack Privacy Preserving

On Log-Loss Scores and (No) Privacy

2020-11-01 · EMNLP (PrivateNLP) 2020 11 · Abhinav Aggarwal, Zekun Xu, Oluwaseyi Feyisetan, Nathanael Teissier

A common metric for assessing the performance of binary classifiers is the Log-Loss score, which is a real number indicating the cross entropy distance between the predicted distribution over the labels and the true dist…

Explanations Leak: Membership Inference with Differential Privacy and Active Learning Defense

2026-02-03 · Fatima Ezzeddine, Osama Zammar, Silvia Giordano, Omran Ayoub arxiv

Counterfactual explanations (CFs) are increasingly integrated into Machine Learning as a Service (MLaaS) systems to improve transparency; however, ML models deployed via APIs are already vulnerable to privacy attacks suc…

Model extraction Active Learning

Unveiling the Unseen: Exploring Whitebox Membership Inference through the Lens of Explainability

2024-07-01 · Chenxi Li, Abhinav Kumar, Zhen Guo, Jie Hou 외

The increasing prominence of deep learning applications and reliance on personalized data underscore the urgent need to address privacy vulnerabilities, particularly Membership Inference Attacks (MIAs). Despite numerous …