An Inversion-based Measure of Memorization for Diffusion Models
The past few years have witnessed substantial advances in image generation powered by diffusion models. However, it was shown that diffusion models are vulnerable to training data memorization, raising concerns regarding copyright infringement and privacy invasion. This study delves into a rigorous analysis of memorization in diffusion models. We introduce an inversion-based measure of memorization, InvMM, which searches for a sensitive latent noise distribution accounting for the replication of an image. For accurate estimation of the memorization score, we propose an adaptive algorithm that balances the normality and sensitivity of the inverted distribution. Comprehensive experiments, conducted on both unconditional and text-guided diffusion models, demonstrate that InvMM is capable of detecting heavily memorized images and elucidating the effect of various factors on memorization. Additionally, we discuss how memorization differs from membership. In practice, InvMM serves as a useful tool for model developers to reliably assess the risk of memorization, thereby contributing to the enhancement of trustworthiness and privacy-preserving capabilities of diffusion models.
Code (1)
Tasks
Image GenerationMemorizationPrivacy PreservingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Latent Diffusion Inversion Requires Understanding the Latent Space
The recovery of training data from generative models ("model inversion") has been extensively studied for diffusion models in the data domain as a memorization/overfitting phenomenon. Latent diffusion models (LDMs), whic…
On the role of memorization in learned priors for geophysical inverse problems
Learned priors based on deep generative models offer data-driven regularization for seismic inversion, but training them requires a dataset of representative subsurface models -- a resource that is inherently scarce in g…
Memorization, Emergence, and Explaining Reversal Failures: A Controlled Study of Relational Semantics in LLMs
Autoregressive LLMs perform well on relational tasks that require linking entities via relational words (e.g., father/son, friend), but it is unclear whether they learn the logical semantics of such relations (e.g., symm…
Filtering Memorization from Parameter-Space in Diffusion Models
Low-Rank Adaptation (LoRA) has become a widely used mechanism for customizing diffusion models, enabling users to inject new visual concepts or styles through lightweight parameter updates. However, LoRAs can memorize tr…
On the Memorization of Consistency Distillation for Diffusion Models
Diffusion models are central to modern generative modeling, and understanding how they balance memorization and generalization is critical for reliable deployment. Recent work has shown that memorization in diffusion mod…