Resource-Constrained Federated Learning with Heterogeneous Labels and Models
Various IoT applications demand resource-constrained machine learning mechanisms for different applications such as pervasive healthcare, activity monitoring, speech recognition, real-time computer vision, etc. This necessitates us to leverage information from multiple devices with few communication overheads. Federated Learning proves to be an extremely viable option for distributed and collaborative machine learning. Particularly, on-device federated learning is an active area of research, however, there are a variety of challenges in addressing statistical (non-IID data) and model heterogeneities. In addition, in this paper we explore a new challenge of interest -- to handle label heterogeneities in federated learning. To this end, we propose a framework with simple $\alpha$-weighted federated aggregation of scores which leverages overlapping information gain across labels, while saving bandwidth costs in the process. Empirical evaluation on Animals-10 dataset (with 4 labels for effective elucidation of results) indicates an average deterministic accuracy increase of at least ~16.7%. We also demonstrate the on-device capabilities of our proposed framework by experimenting with federated learning and inference across different iterations on a Raspberry Pi 2, a single-board computing platform.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningFederated Learningspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization
Devices participating in federated learning (FL) typically have heterogeneous communication, computation, and memory resources. However, in synchronous FL, all devices need to finish training by the same deadline dictate…
FairnessFederated LearningQuantizationAdapterFL: Adaptive Heterogeneous Federated Learning for Resource-constrained Mobile Computing Systems
Federated Learning (FL) enables collaborative learning of large-scale distributed clients without data sharing. However, due to the disparity of computing resources among massive mobile computing devices, the performance…
Federated LearningFADE: Enabling Federated Adversarial Training on Heterogeneous Resource-Constrained Edge Devices
Federated adversarial training can effectively complement adversarial robustness into the privacy-preserving federated learning systems. However, the high demand for memory capacity and computing power makes large-scale …
Adversarial RobustnessFederated LearningPrivacy PreservingFedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models
Federated learning enables multiple distributed devices to collaboratively learn a shared prediction model without centralizing their on-device data. Most of the current algorithms require comparable individual efforts f…
Federated LearningTransfer LearningFedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices
Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional…
Edge-computingFederated Learning