Multi-Agent Deep Reinforcement Learning for Cost- and Delay-Sensitive Virtual Network Function Placement and Routing

2022-06-24 · Shaoyang Wang, Chau Yuen, Wei Ni, Guan Yong Liang, Tiejun Lv

This paper proposes an effective and novel multiagent deep reinforcement learning (MADRL)-based method for solving the joint virtual network function (VNF) placement and routing (P&R), where multiple service requests with differentiated demands are delivered at the same time. The differentiated demands of the service requests are reflected by their delay- and cost-sensitive factors. We first construct a VNF P&R problem to jointly minimize a weighted sum of service delay and resource consumption cost, which is NP-complete. Then, the joint VNF P&R problem is decoupled into two iterative subtasks: placement subtask and routing subtask. Each subtask consists of multiple concurrent parallel sequential decision processes. By invoking the deep deterministic policy gradient method and multi-agent technique, an MADRL-P&R framework is designed to perform the two subtasks. The new joint reward and internal rewards mechanism is proposed to match the goals and constraints of the placement and routing subtasks. We also propose the parameter migration-based model-retraining method to deal with changing network topologies. Corroborated by experiments, the proposed MADRL-P&R framework is superior to its alternatives in terms of service cost and delay, and offers higher flexibility for personalized service demands. The parameter migration-based model-retraining method can efficiently accelerate convergence under moderate network topology changes.

📄 PDF Abstract BibTeX arXiv:2206.12146

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learning

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

DACOM: Learning Delay-Aware Communication for Multi-Agent Reinforcement Learning

2022-12-03 · Tingting Yuan, Hwei-Ming Chung, Jie Yuan, XiaoMing Fu

Communication is supposed to improve multi-agent collaboration and overall performance in cooperative Multi-agent reinforcement learning (MARL). However, such improvements are prevalently limited in practice since most e…

Autonomous Driving Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Communication Gain and Delay Cost Under Cross-Timestep Delays in Cooperative Multi-Agent Reinforcement Learning

2026-04-04 · Zihong Gao, Hongjian Liang, Lei Hao, Liangjun Ke arxiv

Communication is essential for coordination in \emph{cooperative} multi-agent reinforcement learning under partial observability, yet \emph{cross-timestep} delays cause messages to arrive multiple timesteps after generat…

Multi-agent Reinforcement Learning

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

2021-05-10 · Mohammad Akbari, Mohammad Reza Abedi, Roghayeh Joda, Mohsen Pourghasemian 외

In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provide…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Scheduling

Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs

2024-12-18 · Jiaming Yu, Le Liang, Chongtao Guo, Ziyang Guo 외

This paper investigates the use of multi-agent reinforcement learning (MARL) to address distributed channel access in wireless local area networks. In particular, we consider the challenging yet more practical case where…

Collision Avoidance Fairness Multi-agent Reinforcement Learning reinforcement-learning+1

Application of Soft Actor-Critic Algorithms in Optimizing Wastewater Treatment with Time Delays Integration

2024-11-27 · Esmaeel Mohammadi, Daniel Ortiz-Arroyo, Aviaja Anna Hansen, Mikkel Stokholm-Bjerregaard 외

Wastewater treatment plants face unique challenges for process control due to their complex dynamics, slow time constants, and stochastic delays in observations and actions. These characteristics make conventional contro…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning