Federated Multi-Agent Actor-Critic Learning for Age Sensitive Mobile Edge Computing
As an emerging technique, mobile edge computing (MEC) introduces a new processing scheme for various distributed communication-computing systems such as industrial Internet of Things (IoT), vehicular communication, smart city, etc. In this work, we mainly focus on the timeliness of the MEC systems where the freshness of the data and computation tasks is significant. Firstly, we formulate a kind of age-sensitive MEC models and define the average age of information (AoI) minimization problems of interests. Then, a novel policy based multi-agent deep reinforcement learning (RL) framework, called heterogeneous multi-agent actor critic (H-MAAC), is proposed as a paradigm for joint collaboration in the investigated MEC systems, where edge devices and center controller learn the interactive strategies through their own observations. To improves the system performance, we develop the corresponding online algorithm by introducing an edge federated learning mode into the multi-agent cooperation whose advantages on learning convergence can be guaranteed theoretically. To the best of our knowledge, it's the first joint MEC collaboration algorithm that combines the edge federated mode with the multi-agent actor-critic reinforcement learning. Furthermore, we evaluate the proposed approach and compare it with classical RL based methods. As a result, the proposed framework not only outperforms the baseline on average system age, but also promotes the stability of training process. Besides, the simulation results provide some innovative perspectives for the system design under the edge federated collaboration.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningEdge-computingFederated Learningreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Single-Loop Federated Actor-Critic across Heterogeneous Environments
Federated reinforcement learning (FRL) has emerged as a promising paradigm, enabling multiple agents to collaborate and learn a shared policy adaptable across heterogeneous environments. Among the various reinforcement l…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)FedFormer: Contextual Federation with Attention in Reinforcement Learning
A core issue in multi-agent federated reinforcement learning is defining how to aggregate insights from multiple agents. This is commonly done by taking the average of each participating agent's model weights into one co…
Federated Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Enhancing Cyber Resilience of Networked Microgrids using Vertical Federated Reinforcement Learning
This paper presents a novel federated reinforcement learning (Fed-RL) methodology to enhance the cyber resiliency of networked microgrids. We formulate a resilient reinforcement learning (RL) training setup which (a) gen…
OpenAI Gymreinforcement-learningReinforcement LearningReinforcement Learning (RL)FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems
Edge computing addresses the growing data demands of connected-device networks by placing computational resources closer to end users through decentralized infrastructures. This decentralization challenges traditional, f…
Edge-computingFedCM: A Real-time Contribution Measurement Method for Participants in Federated Learning
Federated Learning (FL) creates an ecosystem for multiple agents to collaborate on building models with data privacy consideration. The method for contribution measurement of each agent in the FL system is critical for f…
Federated Learning