LingoQA: Visual Question Answering for Autonomous Driving
We introduce LingoQA, a novel dataset and benchmark for visual question answering in autonomous driving. The dataset contains 28K unique short video scenarios, and 419K annotations. Evaluating state-of-the-art vision-language models on our benchmark shows that their performance is below human capabilities, with GPT-4V responding truthfully to 59.6% of the questions compared to 96.6% for humans. For evaluation, we propose a truthfulness classifier, called Lingo-Judge, that achieves a 0.95 Spearman correlation coefficient to human evaluations, surpassing existing techniques like METEOR, BLEU, CIDEr, and GPT-4. We establish a baseline vision-language model and run extensive ablation studies to understand its performance. We release our dataset and benchmark as an evaluation platform for vision-language models in autonomous driving.
Code (2)
Tasks
Autonomous DrivingDecision MakingLanguage ModelingLanguage ModellingQuestion AnsweringVideo Question AnsweringVisual Question AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AutoDrive-QA- Automated Generation of Multiple-Choice Questions for Autonomous Driving Datasets Using Large Vision-Language Models
In autonomous driving, open-ended question answering often suffers from unreliable evaluations because freeform responses require either complex metrics or subjective human judgment. To address this challenge, we introdu…
Autonomous DrivingMultiple-choiceOpen-Ended Question AnsweringPrediction+1V3LMA: Visual 3D-enhanced Language Model for Autonomous Driving
Large Vision Language Models (LVLMs) have shown strong capabilities in understanding and analyzing visual scenes across various domains. However, in the context of autonomous driving, their limited comprehension of 3D en…
Autonomous DrivingDecision MakingLanguage ModelingLanguage Modelling+1Explaining Autonomous Driving Actions with Visual Question Answering
The end-to-end learning ability of self-driving vehicles has achieved significant milestones over the last decade owing to rapid advances in deep learning and computer vision algorithms. However, as autonomous driving te…
Autonomous DrivingAutonomous VehiclesDecision MakingQuestion Answering+3Efficient Visual Question Answering Pipeline for Autonomous Driving via Scene Region Compression
Autonomous driving increasingly relies on Visual Question Answering (VQA) to enable vehicles to understand complex surroundings by analyzing visual inputs and textual queries. Currently, a paramount concern for VQA in th…
Visual Question AnsweringVideo Question AnsweringComputational EfficiencyAutonomous DrivingNuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario
We introduce a novel visual question answering (VQA) task in the context of autonomous driving, aiming to answer natural language questions based on street-view clues. Compared to traditional VQA tasks, VQA in autonomous…
Autonomous DrivingQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)