Canonicalized Stable-List Replay for Private Federated Continual Learning over Language-Model Embeddings

2026-05-29 · Ibne Farabi Shihab, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Anuj Sharma arxiv

Federated continual learning (FCL) lets distributed clients adapt language-model heads to evolving NLP tasks without sharing raw text. Under user-level differential privacy (DP), replay-based continual learning faces a structural obstacle: clients can release only small noisy lists of candidate replay summaries, and those lists are unordered across clients. We introduce Canonicalized Stable-List Replay (CSLR), where clients privately produce candidate replay distributions over a shared sentence-embedding space and the server aligns them using signatures induced by public anchor sentences. The anchors provide identifiability for aggregation rather than additional replay data. We prove that, under an observable anchor-signature margin, $O(\log(N/η)/p)$ anchors distinguish $N$ candidate list elements with probability at least $1-η$, and we give a scoped anchorless non-identifiability result for unordered-label oracle models. Across five seeds on continual classification, NER, and dialogue benchmarks, CSLR improves the final average task metric by 3.9--5.6 points over the strongest non-CSLR DP baseline at $\eps=4$ under the reported replay-release budget, while also outperforming Hungarian and optimal-transport matchers. The formal privacy guarantee covers replay release; end-to-end private training additionally requires composition with a private optimizer for task-head updates.

📄 PDF Abstract BibTeX arXiv:2606.00426

Code (0)

등록된 구현이 없습니다.

Tasks

Continual Learning

Coordinated Replay Sample Selection for Continual Federated Learning

2023-10-23 · Jack Good, Jimit Majmudar, Christophe Dupuy, Jixuan Wang 외

Continual Federated Learning (CFL) combines Federated Learning (FL), the decentralized learning of a central model on a number of client devices that may not communicate their data, and Continual Learning (CL), the learn…

Continual Learning Federated Learning

Better Generative Replay for Continual Federated Learning

2023-02-25 · Daiqing Qi, Handong Zhao, Sheng Li

Federated learning is a technique that enables a centralized server to learn from distributed clients via communications without accessing the client local data. However, existing federated learning works mainly focus on…

Continual Learning Federated Learning

FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning

2026-07-30 · Zekai Chen, Haodong Lu, Shihao Li, Weiwei Ji 외 arxiv

Federated graph learning enables collaborative training over decentralized graph data without sharing raw graph information. As such risks evolve, clients must learn emerging classes from private multimodal graph streams…

Graph Learning

Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning

2024-09-02 · Jinglin Liang, Jin Zhong, Hanlin Gu, Zhongqi Lu 외

Federated Class Continual Learning (FCCL) merges the challenges of distributed client learning with the need for seamless adaptation to new classes without forgetting old ones. The key challenge in FCCL is catastrophic f…

Continual Learning Contrastive Learning Data-free Knowledge Distillation Domain Generalization+1

Federated Continual Learning for Privacy-Preserving Hospital Imaging Classification

2026-01-11 · Anay Sinhal, Arpana Sinhal, Amit Sinhal arxiv

Deep learning models for radiology interpretation increasingly rely on multi-institutional data, yet privacy regulations and distribution shift across hospitals limit central data pooling. Federated learning (FL) allows …

Federated Learning Continual Learning