Hierarchical Deep Reinforcement Learning Framework for Multi-Year Asset Management Under Budget Constraints

2025-07-25 · Amir Fard, Arnold X. -X. Yuan arxiv

Budget planning and maintenance optimization are crucial for infrastructure asset management, ensuring cost-effectiveness and sustainability. However, the complexity arising from combinatorial action spaces, diverse asset deterioration, stringent budget constraints, and environmental uncertainty significantly limits existing methods' scalability. This paper proposes a Hierarchical Deep Reinforcement Learning methodology specifically tailored to multi-year infrastructure planning. Our approach decomposes the problem into two hierarchical levels: a high-level Budget Planner allocating annual budgets within explicit feasibility bounds, and a low-level Maintenance Planner prioritizing assets within the allocated budget. By structurally separating macro-budget decisions from asset-level prioritization and integrating linear programming projection within a hierarchical Soft Actor-Critic framework, the method efficiently addresses exponential growth in the action space and ensures rigorous budget compliance. A case study evaluating sewer networks of varying sizes (10, 15, and 20 sewersheds) illustrates the effectiveness of the proposed approach. Compared to conventional Deep Q-Learning and enhanced genetic algorithms, our methodology converges more rapidly, scales effectively, and consistently delivers near-optimal solutions even as network size grows.

📄 PDF Abstract BibTeX arXiv:2507.19458

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning

2023-01-25 · Weiguang Han, Boyi Zhang, Qianqian Xie, Min Peng 외

Pair trading is one of the most effective statistical arbitrage strategies which seeks a neutral profit by hedging a pair of selected assets. Existing methods generally decompose the task into two separate steps: pair se…

Hierarchical Reinforcement Learning PAIR TRADING reinforcement-learning Reinforcement Learning (RL)

Multi-Year Maintenance Planning for Large-Scale Infrastructure Systems: A Novel Network Deep Q-Learning Approach

2025-07-24 · Amir Fard, Arnold X. -X. Yuan arxiv

Infrastructure asset management is essential for sustaining the performance of public infrastructure such as road networks, bridges, and utility networks. Traditional maintenance and rehabilitation planning methods often…

Reinforcement Learning

Hierarchical Reinforcement Learning with Deep Nested Agents

2018-05-18 · Marc Brittain, Peng Wei

Deep hierarchical reinforcement learning has gained a lot of attention in recent years due to its ability to produce state-of-the-art results in challenging environments where non-hierarchical frameworks fail to learn us…

Hierarchical Reinforcement Learning Minecraft reinforcement-learning Reinforcement Learning+1

Learning Financial Asset-Specific Trading Rules via Deep Reinforcement Learning

2020-10-27 · Mehran Taghian, Ahmad Asadi, Reza Safabakhsh

Generating asset-specific trading signals based on the financial conditions of the assets is one of the challenging problems in automated trading. Various asset trading rules are proposed experimentally based on differen…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Moira: Language-driven Hierarchical Reinforcement Learning for Pair Trading

2026-05-03 · Polydoros Giannouris, Yuechen Jiang, Lingfei Qian, Yuyan Wang 외 arxiv

Many sequential decision-making problems exhibit hierarchical structure, where high-level semantic choices constrain downstream actions and feedback is delayed and ambiguous. Learning in such settings is challenging due …

Hierarchical Reinforcement Learning PAIR TRADING