Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning
Federated learning (FL) is a framework for machine learning across heterogeneous client devices in a privacy-preserving fashion. To date, most FL algorithms learn a "global" server model across multiple rounds. At each round, the same server model is broadcast to all participating clients, updated locally, and then aggregated across clients. In this work, we propose a more general procedure in which clients "select" what values are sent to them. Notably, this allows clients to operate on smaller, data-dependent slices. In order to make this practical, we outline a primitive, federated select, which enables client-specific selection in realistic FL systems. We discuss how to use federated select for model training and show that it can lead to drastic reductions in communication and client memory usage, potentially enabling the training of models too large to fit on-device. We also discuss the implications of federated select on privacy and trust, which in turn affect possible system constraints and design. Finally, we discuss open questions concerning model architectures, privacy-preserving technologies, and practical FL systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
NCAirFL: CSI-Free Over-the-Air Federated Learning Based on Non-Coherent Detection
Over-the-air federated learning (FL), i.e., AirFL, leverages computing primitively over multiple access channels. A long-standing challenge in AirFL is to achieve coherent signal alignment without relying on expensive ch…
Federated LearningHardware-Aware Federated Learning for Speech Emotion Recognition
Federated learning (FL) enables privacy-preserving collaborative training across distributed edge devices, but real deployments involve heterogeneous clients with different processing power, memory capacity, and communic…
Speech Emotion RecognitionFederated LearningRobust Federated Finetuning of Foundation Models via Alternating Minimization of LoRA
Parameter-Efficient Fine-Tuning (PEFT) has risen as an innovative training strategy that updates only a select few model parameters, significantly lowering both computational and memory demands. PEFT also helps to decrea…
Federated Learningparameter-efficient fine-tuningReconciling Security and Communication Efficiency in Federated Learning
Cross-device Federated Learning is an increasingly popular machine learning setting to train a model by leveraging a large population of client devices with high privacy and security guarantees. However, communication ef…
Federated LearningQuantizationFederated Automatic Differentiation
Federated learning (FL) is a general framework for learning across an axis of group partitioned data (heterogeneous clients) while preserving data privacy, under the orchestration of a central server. FL methods often co…
FADFederated LearningPrivacy Preserving