Variance-Based Membership Inference Attacks Against Large-Scale Image Captioning Models
The proliferation of multi-modal generative models has introduced new privacy and security challenges, especially due to the risks of memorization and unintentional disclosure of sensitive information. This paper focuses on the vulnerability of multi-modal image captioning models to membership inference attacks (MIAs). These models, which synthesize textual descriptions from visual content, could inadvertently reveal personal or proprietary data embedded in their training datasets. We explore the feasibility of MIAs in the context of such models. Specifically, our approach leverages a variance-based strategy tailored for image captioning models, utilizing only image data without knowing the corresponding caption. We introduce the means-of-variance threshold attack (MVTA) and confidence-based weakly supervised attack (C-WSA) based on the metric, means-of-variance (MV), to assess variability among vector embeddings. Our experiments demonstrate that these models are susceptible to MIAs, indicating substantial privacy risks. The effectiveness of our methods is validated through rigorous evaluations on these real-world models, confirming the practical implications of our findings.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningMemorizationSimilar Papers 제목 키워드 기반
NeuGuard: Lightweight Neuron-Guided Defense against Membership Inference Attacks
Membership inference attacks (MIAs) against machine learning models can lead to serious privacy risks for the training dataset used in the model training. In this paper, we propose a novel and effective Neuron-Guided Def…
Reconstruction and Membership Inference Attacks against Generative Models
We present two information leakage attacks that outperform previous work on membership inference against generative models. The first attack allows membership inference without assumptions on the type of the generative m…
Density EstimationInference AttackMembership Inference AttackOne Parameter Defense -- Defending against Data Inference Attacks via Differential Privacy
Machine learning models are vulnerable to data inference attacks, such as membership inference and model inversion attacks. In these types of breaches, an adversary attempts to infer a data record's membership in a datas…
Label-Only Membership Inference Attacks
Membership inference attacks are one of the simplest forms of privacy leakage for machine learning models: given a data point and model, determine whether the point was used to train the model. Existing membership infere…
L2 RegularizationPrivacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning
While being deployed in many critical applications as core components, machine learning (ML) models are vulnerable to various security and privacy attacks. One major privacy attack in this domain is membership inference,…
BIG-bench Machine LearningTransfer Learning