Smart Commander: A Hierarchical Reinforcement Learning Framework for Fleet-Level PHM Decision Optimization
Decision-making in military aviation Prognostics and Health Management (PHM) faces significant challenges due to the "curse of dimensionality" in large-scale fleet operations, combined with sparse feedback and stochastic mission profiles. To address these issues, this paper proposes Smart Commander, a novel Hierarchical Reinforcement Learning (HRL) framework designed to optimize sequential maintenance and logistics decisions. The framework decomposes the complex control problem into a two-tier hierarchy: a strategic General Commander manages fleet-level availability and cost objectives, while tactical Operation Commanders execute specific actions for sortie generation, maintenance scheduling, and resource allocation. The proposed approach is validated within a custom-built, high-fidelity discrete-event simulation environment that captures the dynamics of aircraft configuration and support logistics.By integrating layered reward shaping with planning-enhanced neural networks, the method effectively addresses the difficulty of sparse and delayed rewards. Empirical evaluations demonstrate that Smart Commander significantly outperforms conventional monolithic Deep Reinforcement Learning (DRL) and rule-based baselines. Notably, it achieves a substantial reduction in training time while demonstrating superior scalability and robustness in failure-prone environments. These results highlight the potential of HRL as a reliable paradigm for next-generation intelligent fleet management.
Code (0)
등록된 구현이 없습니다.
Tasks
Hierarchical Reinforcement LearningSimilar Papers 제목 키워드 기반
Analysis of Fleet Modularity in an Artificial Intelligence-Based Attacker-Defender Game
Because combat environments change over time and technology upgrades are widespread for ground vehicles, a large number of vehicles and equipment become quickly obsolete. A possible solution for the U.S. Army is to devel…
Decision MakingEnhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learning
This work presents a Hierarchical Multi-Agent Reinforcement Learning framework for analyzing simulated air combat scenarios involving heterogeneous agents. The objective is to identify effective Courses of Action that le…
Deep Reinforcement LearningMulti-agent Reinforcement Learningreinforcement-learningReinforcement LearningLearning Graph-Enhanced Commander-Executor for Multi-Agent Navigation
This paper investigates the multi-agent navigation problem, which requires multiple agents to reach the target goals in a limited time. Multi-agent reinforcement learning (MARL) has shown promising results for solving th…
Hierarchical Reinforcement LearningMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering
The application of artificial intelligence to simulate air-to-air combat scenarios is attracting increasing attention. To date the high-dimensional state and action spaces, the high complexity of situation information (s…
Decision MakingMulti-agent Reinforcement Learningreinforcement-learningReinforcement LearningFireCommander: An Interactive, Probabilistic Multi-agent Environment for Heterogeneous Robot Teams
The purpose of this tutorial is to help individuals use the \underline{FireCommander} game environment for research applications. The FireCommander is an interactive, probabilistic joint perception-action reconnaissance …
Combinatorial Optimizationreinforcement-learningReinforcement Learning (RL)