A hierarchical control framework for autonomous decision-making systems: Integrating HMDP and MPC
This paper proposes a comprehensive hierarchical control framework for autonomous decision-making arising in robotics and autonomous systems. In a typical hierarchical control architecture, high-level decision making is often characterised by discrete state and decision/control sets. However, a rational decision is usually affected by not only the discrete states of the autonomous system, but also the underlying continuous dynamics even the evolution of its operational environment. This paper proposes a holistic and comprehensive design process and framework for this type of challenging problems, from new modelling and design problem formulation to control design and stability analysis. It addresses the intricate interplay between traditional continuous systems dynamics utilized at the low levels for control design and discrete Markov decision processes (MDP) for facilitating high-level decision making. We model the decision making system in complex environments as a hybrid system consisting of a controlled MDP and autonomous (i.e. uncontrolled) continuous dynamics. Consequently, the new formulation is called as hybrid Markov decision process (HMDP). The design problem is formulated with a focus on ensuring both safety and optimality while taking into account the influence of both the discrete and continuous state variables of different levels. With the help of the model predictive control (MPC) concept, a decision maker design scheme is proposed for the proposed hybrid decision making model. By carefully designing key ingredients involved in this scheme, it is shown that the recursive feasibility and stability of the proposed autonomous decision making scheme are guaranteed. The proposed framework is applied to develop an autonomous lane changing system for intelligent vehicles.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingModel Predictive ControlMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learning
Autonomous driving is a promising technology to reduce traffic accidents and improve driving efficiency. In this work, a deep reinforcement learning (DRL)-enabled decision-making policy is constructed for autonomous vehi…
Autonomous DrivingAutonomous VehiclesDecision MakingDeep Reinforcement Learning+2Hierarchical Decision-Making for Autonomous Navigation: Integrating Deep Reinforcement Learning and Fuzzy Logic in Four-Wheel Independent Steering and Driving Systems
This paper presents a hierarchical decision-making framework for autonomous navigation in four-wheel independent steering and driving (4WISD) systems. The proposed approach integrates deep reinforcement learning (DRL) fo…
Reinforcement LearningCHARMS: A Cognitive Hierarchical Agent for Reasoning and Motion Stylization in Autonomous Driving
To address the challenge of insufficient interactivity and behavioral diversity in autonomous driving decision-making, this paper proposes a Cognitive Hierarchical Agent for Reasoning and Motion Stylization (CHARMS). By …
Autonomous DrivingDecision MakingDeep Reinforcement LearningDiversityReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
End-to-end autonomous driving has emerged as a promising approach to unify perception, prediction, and planning within a single framework, reducing information loss and improving adaptability. However, existing methods o…
Autonomous DrivingHierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization
Hierarchical decision-making frameworks are pivotal for addressing complex control tasks, enabling agents to decompose intricate problems into manageable subgoals. Despite their promise, existing hierarchical policies fa…
Reinforcement LearningCollision AvoidanceDecision Making