Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering
Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Explanation GenerationQuestion AnsweringSimilar Papers 제목 키워드 기반
RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams
The intersection of AI and legal systems presents a growing need for tools that support legal education, particularly in under-resourced languages such as Romanian. In this work, we aim to evaluate the capabilities of La…
Information RetrievalQuestion AnsweringGRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs
LLMs (Large language models) have revolutionized NLP (Natural Language Processing), yet their pedagogical value for low-resource languages remains unclear. We present GRILE (Grammar Romanian Inference and Language Explan…
Explanation GenerationHybrid Reasoning Based on Large Language Models for Autonomous Car Driving
Large Language Models (LLMs) have garnered significant attention for their ability to understand text and images, generate human-like text, and perform complex reasoning tasks. However, their ability to generalize this a…
Autonomous DrivingAutonomous VehiclesCommon Sense ReasoningDecision MakingRoBiologyDataChoiceQA: A Romanian Dataset for improving Biology understanding of Large Language Models
In recent years, large language models (LLMs) have demonstrated significant potential across various natural language processing (NLP) tasks. However, their performance in domain-specific applications and non-English lan…
Prompt EngineeringParameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
Focusing on low-resource languages is an essential step toward democratizing generative AI. In this work, we contribute to reducing the multimodal NLP resource gap for Romanian. We translate the widely known Flickr30K da…
Visual Question Answering