Maneuver Decision-Making Through Proximal Policy Optimization And Monte Carlo Tree Search
Maneuver decision-making can be regarded as a Markov decision process and can be address by reinforcement learning. However, original reinforcement learning algorithms can hardly solve the maneuvering decision-making problem. One reason is that agents use random actions in the early stages of training, which makes it difficult to get rewards and learn how to make effective decisions. To address this issue, a method based on proximal policy optimization and Monte Carlo tree search is proposed. The method uses proximal policy optimization to train the agent, and regards the results of air combat as targets to train the value network. Then, based on the value network and the visit count of each node, Monte Carlo tree search is used to find the actions with more expected returns than random actions, which can improve the training performance. The ablation studies and simulation experiments indicate that agents trained by the proposed method can make different decisions according to different states, which demonstrates that the method can solve the maneuvering decision problem that the original reinforcement learning algorithm cannot solve.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision Makingreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement Learning with Continuous Action Horizon
Decision-making strategy for autonomous vehicles de-scribes a sequence of driving maneuvers to achieve a certain navigational mission. This paper utilizes the deep reinforcement learning (DRL) method to address the conti…
Autonomous VehiclesDecision MakingDeep Reinforcement LearningReinforcement Learning (RL)A Markov Decision Process Framework for Early Maneuver Decisions in Satellite Collision Avoidance
We develop a Markov decision process (MDP) framework to autonomously make guidance decisions for satellite collision avoidance maneuver (CAM) and a reinforcement learning policy gradient (RL-PG) algorithm to enable direc…
Reinforcement LearningCollision AvoidanceGeneralized Proximal Policy Optimization with Sample Reuse
In real-world decision making tasks, it is critical for data-driven reinforcement learning methods to be both stable and sample efficient. On-policy methods typically generate reliable policy improvement throughout train…
Decision MakingResearch on Autonomous Maneuvering Decision of UCAV based on Approximate Dynamic Programming
Unmanned aircraft systems can perform some more dangerous and difficult missions than manned aircraft systems. In some highly complicated and changeable tasks, such as air combat, the maneuvering decision mechanism is re…
Decision Makingfeature selectionAutomated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning
Lane-change maneuvers are commonly executed by drivers to follow a certain routing plan, overtake a slower vehicle, adapt to a merging lane ahead, etc. However, improper lane change behaviors can be a major cause of traf…
Autonomous DrivingDeep Reinforcement Learningreinforcement-learningReinforcement Learning+1