Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars

2022-06-21 · Mingze Wang, Ziyang Zhang, Grace Hui Yang

This paper presents a novel approach that supports natural language voice instructions to guide deep reinforcement learning (DRL) algorithms when training self-driving cars. DRL methods are popular approaches for autonomous vehicle (AV) agents. However, most existing methods are sample- and time-inefficient and lack a natural communication channel with the human expert. In this paper, how new human drivers learn from human coaches motivates us to study new ways of human-in-the-loop learning and a more natural and approachable training interface for the agents. We propose incorporating natural language voice instructions (NLI) in model-based deep reinforcement learning to train self-driving cars. We evaluate the proposed method together with a few state-of-the-art DRL methods in the CARLA simulator. The results show that NLI can help ease the training process and significantly boost the agents' learning speed.

📄 PDF Abstract BibTeX arXiv:2206.10249

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Self-Driving Cars

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

CARLA CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban…

Self-Interest and Systemic Benefits: Emergence of Collective Rationality in Mixed Autonomy Traffic Through Deep Reinforcement Learning

2025-11-07 · Di Chen, Jia Li, Michael Zhang arxiv

Autonomous vehicles (AVs) are expected to be commercially available in the near future, leading to mixed autonomy traffic consisting of both AVs and human-driven vehicles (HVs). Although numerous studies have shown that …

Reinforcement Learning Autonomous Vehicles Federated Learning Decision Making

Conditional Driving from Natural Language Instructions

2019-10-16 · Junha Roh, Chris Paxton, Andrzej Pronobis, Ali Farhadi 외

Widespread adoption of self-driving cars will depend not only on their safety but largely on their ability to interact with human users. Just like human drivers, self-driving cars will be expected to understand and safel…

Imitation Learning Self-Driving Cars

Deep Reinforcement Learning for Human-Like Driving Policies in Collision Avoidance Tasks of Self-Driving Cars

2020-06-07 · Ran Emuna, Avinoam Borowsky, Armin Biess

The technological and scientific challenges involved in the development of autonomous vehicles (AVs) are currently of primary interest for many automobile companies and research labs. However, human-controlled vehicles a…

Autonomous Vehicles Collision Avoidance Deep Reinforcement Learning Gaussian Processes+2

SelfVC: Voice Conversion With Iterative Refinement using Self Transformations

2023-10-14 · Paarth Neekhara, Shehzeen Hussain, Rafael Valle, Boris Ginsburg 외

We propose SelfVC, a training strategy to iteratively improve a voice conversion model with self-synthesized examples. Previous efforts on voice conversion focus on factorizing speech into explicitly disentangled represe…

Self-Supervised Learning Speaker Verification Speech Synthesis Voice Conversion

GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot

2024-12-03 · Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang 외

We introduce GLM-4-Voice, an intelligent and human-like end-to-end spoken chatbot. It supports both Chinese and English, engages in real-time voice conversations, and varies vocal nuances such as emotion, intonation, spe…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Chatbot Language Modeling+6