Privacy-Preserving Serverless Edge Learning with Decentralized Small Data

2021-11-29 · Shih-Chun Lin, Chia-Hung Lin

In the last decade, data-driven algorithms outperformed traditional optimization-based algorithms in many research areas, such as computer vision, natural language processing, etc. However, extensive data usages bring a new challenge or even threat to deep learning algorithms, i.e., privacy-preserving. Distributed training strategies have recently become a promising approach to ensure data privacy when training deep models. This paper extends conventional serverless platforms with serverless edge learning architectures and provides an efficient distributed training framework from the networking perspective. This framework dynamically orchestrates available resources among heterogeneous physical units to efficiently fulfill deep learning objectives. The design jointly considers learning task requests and underlying infrastructure heterogeneity, including last-mile transmissions, computation abilities of mobile devices, edge and cloud computing centers, and devices battery status. Furthermore, to significantly reduce distributed training overheads, small-scale data training is proposed by integrating with a general, simple data classifier. This low-load enhancement can seamlessly work with various distributed deep models to improve communications and computation efficiencies during the training phase. Finally, open challenges and future research directions encourage the research community to develop efficient distributed deep learning techniques.

📄 PDF Abstract BibTeX arXiv:2111.14955

Code (0)

등록된 구현이 없습니다.

Tasks

Cloud ComputingDeep LearningPrivacy Preserving

Hierarchical Decentralized Multi-Agent Coordination with Privacy-Preserving Knowledge Sharing: Extending AgentNet for Scalable Autonomous Systems

2025-11-29 · Goutham Nalagatla arxiv

Decentralized multi-agent systems have shown promise in enabling autonomous collaboration among LLM-based agents. While AgentNet demonstrated the feasibility of fully decentralized coordination through dynamic DAG topolo…

Knowledge Distillation

Adaptive Serverless Learning

2020-08-24 · Hongchang Gao, Heng Huang

With the emergence of distributed data, training machine learning models in the serverless manner has attracted increasing attention in recent years. Numerous training approaches have been proposed in this regime, such a…

Towards Private Learning on Decentralized Graphs with Local Differential Privacy

2022-01-23 · WanYu Lin, Baochun Li, Cong Wang

Many real-world networks are inherently decentralized. For example, in social networks, each user maintains a local view of a social graph, such as a list of friends and her profile. It is typical to collect these local …

Benchmarking Graph Learning Privacy Preserving

Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors

2024-05-19 · Chun-Yin Huang, Kartik Srinivas, Xin Zhang, Xiaoxiao Li

Conventional Federated Learning (FL) involves collaborative training of a global model while maintaining user data privacy. One of its branches, decentralized FL, is a serverless network that allows clients to own and op…

Domain Adaptation Federated Learning Knowledge Distillation Transfer Learning

SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks

2021-06-04 · Chaoyang He, Emir Ceyani, Keshav Balasubramanian, Murali Annavaram 외

Graph Neural Networks (GNNs) are the first choice methods for graph machine learning problems thanks to their ability to learn state-of-the-art level representations from graph-structured data. However, centralizing a ma…

BIG-bench Machine Learning Federated Learning Molecular Property Prediction Multi-Task Learning+1