Communication-Efficient Consensus Mechanism for Federated Reinforcement Learning

2022-01-30 · Xing Xu, Rongpeng Li, Zhifeng Zhao, Honggang Zhang

The paper considers independent reinforcement learning (IRL) for multi-agent decision-making process in the paradigm of federated learning (FL). We show that FL can clearly improve the policy performance of IRL in terms of training efficiency and stability. However, since the policy parameters are trained locally and aggregated iteratively through a central server in FL, frequent information exchange incurs a large amount of communication overheads. To reach a good balance between improving the model's convergence performance and reducing the required communication and computation overheads, this paper proposes a system utility function and develops a consensus-based optimization scheme on top of the periodic averaging method, which introduces the consensus algorithm into FL for the exchange of a model's local gradients. This paper also provides novel convergence guarantees for the developed method, and demonstrates its superior effectiveness and efficiency in improving the system utility value through theoretical analyses and numerical simulation results.

📄 PDF Abstract BibTeX arXiv:2201.12718

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Federated Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence

2023-11-21 · Shu Zheng, Tiandi Ye, Xiang Li, Ming Gao

Federated learning (FL) on heterogeneous data (non-IID data) has recently received great attention. Most existing methods focus on studying the convergence guarantees for the global objective. While these methods can gua…

Fairness Federated Learning

Trust-based Consensus in Multi-Agent Reinforcement Learning Systems

2022-05-25 · Ho Long Fung, Victor-Alexandru Darvariu, Stephen Hailes, Mirco Musolesi

An often neglected issue in multi-agent reinforcement learning (MARL) is the potential presence of unreliable agents in the environment whose deviations from expected behavior can prevent a system from accomplishing its …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

2024-07-11 · Pu Feng, Junkang Liang, Size Wang, Xin Yu 외

In multi-agent reinforcement learning (MARL), the Centralized Training with Decentralized Execution (CTDE) framework is pivotal but struggles due to a gap: global state guidance in training versus reliance on local obser…

Contrastive Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Proof-of-Collaborative-Learning: A Multi-winner Federated Learning Consensus Algorithm

2024-07-17 · Amirreza Sokhankhosh, Sara Rouhani

Regardless of their variations, blockchains require a consensus mechanism to validate transactions, supervise added blocks, maintain network security, synchronize the network state, and distribute incentives. Proof-of-Wo…

Federated Learning

Dialogue Diplomats: An End-to-End Multi-Agent Reinforcement Learning System for Automated Conflict Resolution and Consensus Building

2025-11-20 · Deepak Bolleddu arxiv

Conflict resolution and consensus building represent critical challenges in multi-agent systems, negotiations, and collaborative decision-making processes. This paper introduces Dialogue Diplomats, a novel end-to-end mul…

Multi-agent Reinforcement Learning