FedorAS: Federated Architecture Search under system heterogeneity

2022-06-22 · Lukasz Dudziak, Stefanos Laskaridis, Javier Fernandez-Marques

Federated learning (FL) has recently gained considerable attention due to its ability to learn on decentralised data while preserving client privacy. However, it also poses additional challenges related to the heterogeneity of the participating devices, both in terms of their computational capabilities and contributed data. Meanwhile, Neural Architecture Search (NAS) has been successfully used with centralised datasets, producing state-of-the-art results in constrained or unconstrained settings. However, such centralised datasets may not be always available for training. Most recent work at the intersection of NAS and FL attempts to alleviate this issue in a cross-silo federated setting, which assumes homogeneous compute environments with datacenter-grade hardware. In this paper we explore the question of whether we can design architectures of different footprints in a cross-device federated setting, where the device landscape, availability and scale are very different. To this end, we design our system, FedorAS, to discover and train promising architectures in a resource-aware manner when dealing with devices of varying capabilities holding non-IID distributed data. We present empirical evidence of its effectiveness across different settings, spanning across three different modalities (vision, speech, text), and showcase its better performance compared to state-of-the-art federated solutions, while maintaining resource efficiency.

📄 PDF Abstract BibTeX arXiv:2206.11239

Code (0)

등록된 구현이 없습니다.

Tasks

Federated Learning Neural Architecture Search

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

2020-04-18 · Chaoyang He, Murali Annavaram, Salman Avestimehr

Federated Learning (FL) has been proved to be an effective learning framework when data cannot be centralized due to privacy, communication costs, and regulatory restrictions. When training deep learning models under an …

Federated Learning Neural Architecture Search

A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection

2019-07-23 · Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu 외

Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine …

BIG-bench Machine Learning Federated Learning Privacy Preserving

A Systematic Literature Review on Federated Machine Learning: From A Software Engineering Perspective

2020-07-22 · Sin Kit Lo, Qinghua Lu, Chen Wang, Hye-Young Paik 외

Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. To identify the state-of-the-art in federated learning and exp…

BIG-bench Machine Learning Federated Learning Systematic Literature Review

Decision Models for Selecting Federated Learning Architecture Patterns

2022-04-28 · Sin Kit Lo, Qinghua Lu, Hye-Young Paik, Liming Zhu

Federated machine learning is growing fast in academia and industries as a solution to solve data hungriness and privacy issues in machine learning. Being a widely distributed system, federated machine learning requires …

Federated Learning Management Systematic Literature Review

Designing Sustainable Federated Learning as a Service using Neural Architecture Search

2026-08-14 · Keya Patel, Sajib Mistry, Sheik Fattah, Deepak Kanneganti 외 arxiv

The sustainability constraints of FLaaS consumers pose significant challenges to maintaining carbon-feasible federated training in FLaaS environments. These constraints often lead to infeasible consumer participation and…

Neural Architecture Search Federated Learning