Shaping Rewards, Shaping Routes: On Multi-Agent Deep Q-Networks for Routing in Satellite Constellation Networks

2024-08-04 · Manuel M. H. Roth, Anupama Hegde, Thomas Delamotte, Andreas Knopp

Effective routing in satellite mega-constellations has become crucial to facilitate the handling of increasing traffic loads, more complex network architectures, as well as the integration into 6G networks. To enhance adaptability as well as robustness to unpredictable traffic demands, and to solve dynamic routing environments efficiently, machine learning-based solutions are being considered. For network control problems, such as optimizing packet forwarding decisions according to Quality of Service requirements and maintaining network stability, deep reinforcement learning techniques have demonstrated promising results. For this reason, we investigate the viability of multi-agent deep Q-networks for routing in satellite constellation networks. We focus specifically on reward shaping and quantifying training convergence for joint optimization of latency and load balancing in static and dynamic scenarios. To address identified drawbacks, we propose a novel hybrid solution based on centralized learning and decentralized control.

📄 PDF Abstract BibTeX arXiv:2408.01979

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learning

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

Focus 설명 없음

Shaping Advice in Deep Reinforcement Learning

2022-02-19 · Baicen Xiao, Bhaskar Ramasubramanian, Radha Poovendran

Reinforcement learning involves agents interacting with an environment to complete tasks. When rewards provided by the environment are sparse, agents may not receive immediate feedback on the quality of actions that they…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Shaping Advice in Deep Multi-Agent Reinforcement Learning

2021-03-29 · Baicen Xiao, Bhaskar Ramasubramanian, Radha Poovendran

Multi-agent reinforcement learning involves multiple agents interacting with each other and a shared environment to complete tasks. When rewards provided by the environment are sparse, agents may not receive immediate fe…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

ARMS: Automatic Reward Shaping for Sparse-Reward Multi-Agent Reinforcement Learning

2026-05-22 · Elie Abboud, Oren Gal arxiv

Sparse rewards are a major bottleneck in multi-agent reinforcement learning (MARL), where simultaneous learning induces non-stationarity and makes reward design especially delicate. Reward shaping can accelerate learning…

Multi-agent Reinforcement Learning

Learning to Shape Rewards using a Game of Two Partners

2021-03-16 · David Mguni, Taher Jafferjee, Jianhong Wang, Nicolas Perez-Nieves 외

Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards. However, RS typically relies on manually engineered shaping-reward functions whose co…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Valence Prediction

Environment Shaping in Reinforcement Learning using State Abstraction

2020-06-23 · Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

One of the central challenges faced by a reinforcement learning (RL) agent is to effectively learn a (near-)optimal policy in environments with large state spaces having sparse and noisy feedback signals. In real-world a…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)