MAMRL: Exploiting Multi-agent Meta Reinforcement Learning in WAN Traffic Engineering
Traffic optimization challenges, such as load balancing, flow scheduling, and improving packet delivery time, are difficult online decision-making problems in wide area networks (WAN). Complex heuristics are needed for instance to find optimal paths that improve packet delivery time and minimize interruptions which may be caused by link failures or congestion. The recent success of reinforcement learning (RL) algorithms can provide useful solutions to build better robust systems that learn from experience in model-free settings. In this work, we consider a path optimization problem, specifically for packet routing, in large complex networks. We develop and evaluate a model-free approach, applying multi-agent meta reinforcement learning (MAMRL) that can determine the next-hop of each packet to get it delivered to its destination with minimum time overall. Specifically, we propose to leverage and compare deep policy optimization RL algorithms for enabling distributed model-free control in communication networks and present a novel meta-learning-based framework, MAMRL, for enabling quick adaptation to topology changes. To evaluate the proposed framework, we simulate with various WAN topologies. Our extensive packet-level simulation results show that compared to classical shortest path and traditional reinforcement learning approaches, MAMRL significantly reduces the average packet delivery time even when network demand increases; and compared to a non-meta deep policy optimization algorithm, our results show the reduction of packet loss in much fewer episodes when link failures occur while offering comparable average packet delivery time.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingMeta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)SchedulingSimilar Papers 제목 키워드 기반
Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems
The stringent requirements of mobile edge computing (MEC) applications and functions fathom the high capacity and dense deployment of MEC hosts to the upcoming wireless networks. However, operating such high capacity MEC…
Edge-computingMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1PERIL: Probabilistic Embeddings for hybrid Meta-Reinforcement and Imitation Learning
Imitation learning is a natural way for a human to describe a task to an agent, and it can be combined with reinforcement learning to enable the agent to solve that task through exploration. However, traditional methods …
Imitation LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1Quantum Multi-Agent Meta Reinforcement Learning
Although quantum supremacy is yet to come, there has recently been an increasing interest in identifying the potential of quantum machine learning (QML) in the looming era of practical quantum computing. Motivated by thi…
Meta-LearningMeta Reinforcement LearningMulti-agent Reinforcement LearningQuantum Machine Learning+3Causal Reasoning from Meta-reinforcement Learning
Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent …
counterfactualMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance
Deep reinforcement learning (DRL) has shown huge potentials in building financial market simulators recently. However, due to the highly complex and dynamic nature of real-world markets, raw historical financial data oft…
Deep Reinforcement LearningGPUreinforcement-learningReinforcement Learning+1