Human-centric Reward Optimization for Reinforcement Learning-based Automated Driving using Large Language Models
One of the key challenges in current Reinforcement Learning (RL)-based Automated Driving (AD) agents is achieving flexible, precise, and human-like behavior cost-effectively. This paper introduces an innovative approach that uses large language models (LLMs) to intuitively and effectively optimize RL reward functions in a human-centric way. We developed a framework where instructions and dynamic environment descriptions are input into the LLM. The LLM then utilizes this information to assist in generating rewards, thereby steering the behavior of RL agents towards patterns that more closely resemble human driving. The experimental results demonstrate that this approach not only makes RL agents more anthropomorphic but also achieves better performance. Additionally, various strategies for reward-proxy and reward-shaping are investigated, revealing the significant impact of prompt design on shaping an AD vehicle's behavior. These findings offer a promising direction for the development of more advanced, human-like automated driving systems. Our experimental data and source code can be found here
Code (1)
Tasks
In-Context LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment
AI alignment is growing in importance, yet many current approaches learn safety behavior by directly modifying policy parameters, entangling normative constraints with the underlying policy. This often yields opaque, dif…
Reinforcement LearningEgoVLM: Policy Optimization for Egocentric Video Understanding
Emerging embodied AI applications, such as wearable cameras and autonomous agents, have underscored the need for robust reasoning from first person video streams. We introduce EgoVLM, a vision-language model specifically…
EgoSchemaQuestion Answeringreinforcement-learningReinforcement Learning+2HOPE: Human-Centric Off-Policy Evaluation for E-Learning and Healthcare
Reinforcement learning (RL) has been extensively researched for enhancing human-environment interactions in various human-centric tasks, including e-learning and healthcare. Since deploying and evaluating policies online…
Off-policy evaluationReinforcement Learning (RL)Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF
Reinforcement Learning from Human Feedback (RLHF) is a pivotal technique that aligns language models closely with human-centric values. The initial phase of RLHF involves learning human values using a reward model from r…
Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging task
We develop a simple framework to learn bio-inspired foraging policies using human data. We conduct an experiment where humans are virtually immersed in an open field foraging environment and are trained to collect the hi…
Deep Reinforcement LearningImitation LearningReinforcement Learning (RL)