Mitigating Unintended Memorization with LoRA in Federated Learning for LLMs

2025-02-07 · Thierry Bossy, Julien Vignoud, Tahseen Rabbani, Juan R. Troncoso Pastoriza, Martin Jaggi

Federated learning (FL) is a popular paradigm for collaborative training which avoids direct data exposure between clients. However, data privacy issues still remain: FL-trained large language models are capable of memorizing and completing phrases and sentences contained in training data when given with their prefixes. Thus, it is possible for adversarial and honest-but-curious clients to recover training data of other participants simply through targeted prompting. In this work, we demonstrate that a popular and simple fine-tuning strategy, low-rank adaptation (LoRA), reduces memorization during FL up to a factor of 10. We study this effect by performing a medical question-answering fine-tuning task and injecting multiple replicas of out-of-distribution sensitive sequences drawn from an external clinical dataset. We observe a reduction in memorization for a wide variety of Llama 2 and 3 models, and find that LoRA can reduce memorization in centralized learning as well. Furthermore, we show that LoRA can be combined with other privacy-preserving techniques such as gradient clipping and Gaussian noising, secure aggregation, and Goldfish loss to further improve record-level privacy while maintaining performance.

📄 PDF Abstract BibTeX arXiv:2502.05087

Code (1)

tuneinsight/federated-llms 공식 구현 pytorch

Tasks

Federated LearningMedical Question AnsweringMemorizationPrivacy PreservingQuestion Answering

Methods 이 논문이 사용한 방법론

Gradient Clipping One difficulty that arises with optimization of deep neural networks is that large parameter gradients can lead an SGD optimizer to update…

LLaMA LLaMA is a collection of foundation language models ranging from 7B to 65B parameters. It is based on the transformer architecture with various improvements that were…

Understanding Unintended Memorization in Federated Learning

2020-06-12 · Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, Françoise Beaufays

Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are often trained on sensitive data, to ensur…

Clustering Federated Learning Memorization

Understanding Unintended Memorization in Language Models Under Federated Learning

2021-06-01 · NAACL (PrivateNLP) 2021 6 · Om Dipakbhai Thakkar, Swaroop Ramaswamy, Rajiv Mathews, Francoise Beaufays

Recent works have shown that language models (LMs), e.g., for next word prediction (NWP), have a tendency to memorize rare or unique sequences in the training data. Since useful LMs are often trained on sensitive data, i…

Federated Learning Memorization

Mitigating Memorization in LLMs using Activation Steering

2025-03-08 · Manan Suri, Nishit Anand, Amisha Bhaskar

The memorization of training data by Large Language Models (LLMs) poses significant risks, including privacy leaks and the regurgitation of copyrighted content. Activation steering, a technique that directly intervenes i…

Memorization Privacy Preserving

Mitigating Unintended Memorization in Language Models via Alternating Teaching

2022-10-13 · Zhe Liu, Xuedong Zhang, Fuchun Peng

Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. We employ a teacher-student framework and…

Memorization Privacy Preserving

SoK: Unintended Interactions among Machine Learning Defenses and Risks

2023-12-07 · Vasisht Duddu, Sebastian Szyller, N. Asokan

Machine learning (ML) models cannot neglect risks to security, privacy, and fairness. Several defenses have been proposed to mitigate such risks. When a defense is effective in mitigating one risk, it may correspond to i…

Fairness Memorization