Language and Planning in Robotic Navigation: A Multilingual Evaluation of State-of-the-Art Models
Large Language Models (LLMs) such as GPT-4, trained on huge amount of datasets spanning multiple domains, exhibit significant reasoning, understanding, and planning capabilities across various tasks. This study presents the first-ever work in Arabic language integration within the Vision-and-Language Navigation (VLN) domain in robotics, an area that has been notably underexplored in existing research. We perform a comprehensive evaluation of state-of-the-art multi-lingual Small Language Models (SLMs), including GPT-4o mini, Llama 3 8B, and Phi-3 medium 14B, alongside the Arabic-centric LLM, Jais. Our approach utilizes the NavGPT framework, a pure LLM-based instruction-following navigation agent, to assess the impact of language on navigation reasoning through zero-shot sequential action prediction using the R2R dataset. Through comprehensive experiments, we demonstrate that our framework is capable of high-level planning for navigation tasks when provided with instructions in both English and Arabic. However, certain models struggled with reasoning and planning in the Arabic language due to inherent limitations in their capabilities, sub-optimal performance, and parsing issues. These findings highlight the importance of enhancing planning and reasoning capabilities in language models for effective navigation, emphasizing this as a key area for further development while also unlocking the potential of Arabic-language models for impactful real-world applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Instruction FollowingVision and Language NavigationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation
Although ground robotic autonomy has gained widespread usage in structured and controlled environments, autonomy in unknown and off-road terrain remains a difficult problem. Extreme, off-road, and unstructured environmen…
Autonomous NavigationModel Predictive ControlMotion PlanningGeneral-Purpose Robotic Navigation via LVLM-Orchestrated Perception, Reasoning, and Acting
Developing general-purpose navigation policies for unknown environments remains a core challenge in robotics. Most existing systems rely on task-specific neural networks and fixed data flows, limiting generalizability. L…
Embodied Question AnsweringQuestion AnsweringLearning Social Heuristics for Human-Aware Path Planning
Social robotic navigation has been at the center of numerous studies in recent years. Most of the research has focused on driving the robotic agent along obstacle-free trajectories, respecting social distances from human…
End-to-end Sketch-Guided Path Planning through Imitation Learning for Autonomous Mobile Robots
Path planning is crucial for Autonomous Mobile Robots applications. Traditionally, path planning based on human input and preferences has relied on hard to define reward-based learning or costly techniques requiring addi…
Image GenerationImitation LearningTopological Planning with Transformers for Vision-and-Language Navigation
Conventional approaches to vision-and-language navigation (VLN) are trained end-to-end but struggle to perform well in freely traversable environments. Inspired by the robotics community, we propose a modular approach to…
Vision and Language Navigation