Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills
Learning-based vehicle planning is receiving increasing attention with the emergence of diverse driving simulators and large-scale driving datasets. While offline reinforcement learning (RL) is well suited for these safety-critical tasks, it still struggles to plan over extended periods. In this work, we present a skill-based framework that enhances offline RL to overcome the long-horizon vehicle planning challenge. Specifically, we design a variational autoencoder (VAE) to learn skills from offline demonstrations. To mitigate posterior collapse of common VAEs, we introduce a two-branch sequence encoder to capture both discrete options and continuous variations of the complex driving skills. The final policy treats learned skills as actions and can be trained by any off-the-shelf offline RL algorithms. This facilitates a shift in focus from per-step actions to temporally extended skills, thereby enabling long-term reasoning into the future. Extensive results on CARLA prove that our model consistently outperforms strong baselines at both training and new scenarios. Additional visualizations and experiments demonstrate the interpretability and transferability of extracted skills.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingOffline RLreinforcement-learningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
Offline reinforcement learning has emerged as a promising technology by enhancing its practicality through the use of pre-collected large datasets. Despite its practical benefits, most algorithm development research in o…
Autonomous DrivingDecision Makingreinforcement-learningReinforcement LearningSafety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving
In the domain of autonomous driving, the offline Reinforcement Learning~(RL) approaches exhibit notable efficacy in addressing sequential decision-making problems from offline datasets. However, maintaining safety in div…
Autonomous DrivingAutonomous VehiclesDecision MakingOffline RL+4Boosting Offline Reinforcement Learning with Action Preference Query
Training practical agents usually involve offline and online reinforcement learning (RL) to balance the policy's performance and interaction costs. In particular, online fine-tuning has become a commonly used method to c…
Autonomous DrivingD4RLreinforcement-learningReinforcement Learning+1PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning
End-to-end autonomous driving policies based on Imitation Learning (IL) often struggle in closed-loop execution due to the misalignment between inadequate open-loop training objectives and real driving requirements. Whil…
Reinforcement LearningAutonomous DrivingSpatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving
Despite advances in hierarchical reinforcement learning, its applications to path planning in autonomous driving on highways are challenging. One reason is that conventional hierarchical reinforcement learning approaches…
Autonomous DrivingHierarchical Reinforcement Learningreinforcement-learningReinforcement Learning+1