DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks
This paper proposes DeepRMSA, a deep reinforcement learning framework for routing, modulation and spectrum assignment (RMSA) in elastic optical networks (EONs). DeepRMSA learns the correct online RMSA policies by parameterizing the policies with deep neural networks (DNNs) that can sense complex EON states. The DNNs are trained with experiences of dynamic lightpath provisioning. We first modify the asynchronous advantage actor-critic algorithm and present an episode-based training mechanism for DeepRMSA, namely, DeepRMSA-EP. DeepRMSA-EP divides the dynamic provisioning process into multiple episodes (each containing the servicing of a fixed number of lightpath requests) and performs training by the end of each episode. The optimization target of DeepRMSA-EP at each step of servicing a request is to maximize the cumulative reward within the rest of the episode. Thus, we obviate the need for estimating the rewards related to unknown future states. To overcome the instability issue in the training of DeepRMSA-EP due to the oscillations of cumulative rewards, we further propose a window-based flexible training mechanism, i.e., DeepRMSA-FLX. DeepRMSA-FLX attempts to smooth out the oscillations by defining the optimization scope at each step as a sliding window, and ensuring that the cumulative rewards always include rewards from a fixed number of requests. Evaluations with the two sample topologies show that DeepRMSA-FLX can effectively stabilize the training while achieving blocking probability reductions of more than 20.3% and 14.3%, when compared with the baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
BlockingDeep Reinforcement LearningReinforcement LearningSimilar Papers 제목 키워드 기반
OpticGAI: Generative AI-aided Deep Reinforcement Learning for Optical Networks Optimization
Deep Reinforcement Learning (DRL) is regarded as a promising tool for optical network optimization. However, the flexibility and efficiency of current DRL-based solutions for optical network optimization require further …
BlockingDeep Reinforcement LearningResource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach
A deep reinforcement learning approach is applied, for the first time, to solve the routing, modulation, spectrum and core allocation (RMSCA) problem in dynamic multicore fiber elastic optical networks (MCF-EONs). To do …
BlockingDeep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer
This paper proposes a novel scalable reinforcement learning approach for simultaneous routing and spectrum access in wireless ad-hoc networks. In most previous works on reinforcement learning for network optimization, th…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Combining Contention-Based Spectrum Access and Adaptive Modulation using Deep Reinforcement Learning
The use of unlicensed spectrum for cellular systems to mitigate spectrum scarcity has led to the development of intelligent adaptive approaches to spectrum access that improve upon traditional carrier sensing and listen-…
Deep Reinforcement LearningFairnessreinforcement-learningReinforcement Learning (RL)xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
Modern LLM deployments confront a widening cost-performance spectrum: premium models deliver strong reasoning but are expensive, while lightweight models are economical yet brittle on complex tasks. Static escalation rul…
Reinforcement Learning