Over-the-air Federated Policy Gradient
In recent years, over-the-air aggregation has been widely considered in large-scale distributed learning, optimization, and sensing. In this paper, we propose the over-the-air federated policy gradient algorithm, where all agents simultaneously broadcast an analog signal carrying local information to a common wireless channel, and a central controller uses the received aggregated waveform to update the policy parameters. We investigate the effect of noise and channel distortion on the convergence of the proposed algorithm, and establish the complexities of communication and sampling for finding an $\epsilon$-approximate stationary point. Finally, we present some simulation results to show the effectiveness of the algorithm.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On Global Convergence Rates for Federated Policy Gradient under Heterogeneous Environment
Ensuring convergence of policy gradient methods in federated reinforcement learning (FRL) under environment heterogeneity remains a major challenge. In this work, we first establish that heterogeneity, perhaps counter-in…
Federated LearningPolicy Gradient MethodsQ-LearningFederated Natural Policy Gradient and Actor Critic Methods for Multi-task Reinforcement Learning
Federated reinforcement learning (RL) enables collaborative decision making of multiple distributed agents without sharing local data trajectories. In this work, we consider a multi-task setting, in which each agent has …
Decision MakingPolicy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates
Federated reinforcement learning (FedRL) enables agents to collaboratively train a global policy without sharing their individual data. However, high communication overhead remains a critical bottleneck, particularly for…
MuJoCoScalar Federated Learning for Linear Quadratic Regulator
We propose ScalarFedLQR, a communication-efficient federated algorithm for model-free learning of a common policy in linear quadratic regulator (LQR) control of heterogeneous agents. The method builds on a decomposed pro…
Federated LearningFederated Reinforcement Learning with Constraint Heterogeneity
We study a Federated Reinforcement Learning (FedRL) problem with constraint heterogeneity. In our setting, we aim to solve a reinforcement learning problem with multiple constraints while $N$ training agents are located …
Language ModelingLanguage ModellingLarge Language ModelPolicy Gradient Methods+2