Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data

2018-11-28 · Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, Seong-Lyun Kim

On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communication payload size is much smaller than a benchmark scheme, federated learning (FL), particularly when the model size is large. Moreover, user-generated data samples are likely to become non-IID across devices, which commonly degrades the performance compared to the case with an IID dataset. To cope with this, we propose federated augmentation (FAug), where each device collectively trains a generative model, and thereby augments its local data towards yielding an IID dataset. Empirical studies demonstrate that FD with FAug yields around 26x less communication overhead while achieving 95-98% test accuracy compared to FL.

📄 PDF Abstract BibTeX arXiv:1811.11479

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Federated Learning

FedICT: Federated Multi-task Distillation for Multi-access Edge Computing

2023-01-01 · Zhiyuan Wu, Sheng Sun, Yuwei Wang, Min Liu 외

The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning in Multi-access Edge Computing (MEC). Diverse user behaviors call …

Edge-computing Federated Learning Knowledge Distillation Multi-Task Learning

FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation

2024-05-22 · Quyang Pan, Sheng Sun, Zhiyuan Wu, Yuwei Wang 외

Federated Edge Learning (FEL) has emerged as a promising approach for enabling edge devices to collaboratively train machine learning models while preserving data privacy. Despite its advantages, practical FEL deployment…

Dataset Distillation Federated Learning

Mix2FLD: Downlink Federated Learning After Uplink Federated Distillation With Two-Way Mixup

2020-06-17 · Seungeun Oh, Jihong Park, Eunjeong Jeong, Hyesung Kim 외

This letter proposes a novel communication-efficient and privacy-preserving distributed machine learning framework, coined Mix2FLD. To address uplink-downlink capacity asymmetry, local model outputs are uploaded to a ser…

Federated Learning Privacy Preserving

Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data

2020-08-14 · Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura 외

This study develops a federated learning (FL) framework overcoming largely incremental communication costs due to model sizes in typical frameworks without compromising model performance. To this end, based on the idea o…

Data Augmentation Federated Learning

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

2021-10-19 · Chan Yun Hin, Ngai Edith

Federated learning (FL) is able to manage edge devices to cooperatively train a model while maintaining the training data local and private. One common assumption in FL is that all edge devices share the same machine lea…

Federated Learning Knowledge Distillation