paper-with-me

Papers

Exploring User-level Gradient Inversion with a Diffusion Prior

2024-09-11 · Zhuohang Li, Andrew Lowy, Jing Liu, Toshiaki Koike-Akino, Bradley Malin, Kieran Parsons, Ye Wang

We explore user-level gradient inversion as a new attack surface in distributed learning. We first investigate existing attacks on their ability to make inferences about private information beyond training data reconstruction. Motivated by the low reconstruction quality of existing methods, we propose a novel gradient inversion attack that applies a denoising diffusion model as a strong image prior in order to enhance recovery in the large batch setting. Unlike traditional attacks, which aim to reconstruct individual samples and suffer at large batch and image sizes, our approach instead aims to recover a representative image that captures the sensitive shared semantic information corresponding to the underlying user. Our experiments with face images demonstrate the ability of our methods to recover realistic facial images along with private user attributes.

📄 PDF Abstract BibTeX arXiv:2409.07291

Code (0)

등록된 구현이 없습니다.

Tasks

Denoising

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Gradient-free Decoder Inversion in Latent Diffusion Models

2024-09-27 · Seongmin Hong, Suh Yoon Jeon, Kyeonghyun Lee, Ernest K. Ryu 외

In latent diffusion models (LDMs), denoising diffusion process efficiently takes place on latent space whose dimension is lower than that of pixel space. Decoder is typically used to transform the representation in laten…

DecoderDenoisingGPUScheduling

Gradient Inversion of Federated Diffusion Models

2024-05-30 · Jiyue Huang, Chi Hong, Lydia Y. Chen, Stefanie Roos

Diffusion models are becoming defector generative models, which generate exceptionally high-resolution image data. Training effective diffusion models require massive real data, which is privately owned by distributed pa…

Federated LearningPrivacy Preserving

Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models

2023-12-19 · CVPR 2024 1 · Shweta Mahajan, Tanzila Rahman, Kwang Moo Yi, Leonid Sigal

The quality of the prompts provided to text-to-image diffusion models determines how faithful the generated content is to the user's intent, often requiring `prompt engineering'. To harness visual concepts from target im…

Image GenerationPrompt Engineering

MGIC: A Multi-Label Gradient Inversion Attack based on Canny Edge Detection on Federated Learning

2024-03-13 · Can Liu, Jin Wang

As a new distributed computing framework that can protect data privacy, federated learning (FL) has attracted more and more attention in recent years. It receives gradients from users to train the global model and releas…

Distributed ComputingEdge DetectionFederated LearningImage Reconstruction

Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives

2026-07-29 · Xiaolong Liu, Junjian Li, Yuan Xiao, Jiaqi Deng 외 arxiv

Prompt inversion, as a typical reverse engineering technique, enables text-to-image (T2I) diffusion models to generate the desired target images without extensive prompt engineering. However, existing prompt inversion me…

Prompt EngineeringImage Editing