VisioPath: Vision-Language Enhanced Model Predictive Control for Safe Autonomous Navigation in Mixed Traffic
In this paper, we introduce VisioPath, a novel framework combining vision-language models (VLMs) with model predictive control (MPC) to enable safe autonomous driving in dynamic traffic environments. The proposed approach leverages a bird's-eye view video processing pipeline and zero-shot VLM capabilities to obtain structured information about surrounding vehicles, including their positions, dimensions, and velocities. Using this rich perception output, we construct elliptical collision-avoidance potential fields around other traffic participants, which are seamlessly integrated into a finite-horizon optimal control problem for trajectory planning. The resulting trajectory optimization is solved via differential dynamic programming with an adaptive regularization scheme and is embedded in an event-triggered MPC loop. To ensure collision-free motion, a safety verification layer is incorporated in the framework that provides an assessment of potential unsafe trajectories. Extensive simulations in Simulation of Urban Mobility (SUMO) demonstrate that VisioPath outperforms conventional MPC baselines across multiple metrics. By combining modern AI-driven perception with the rigorous foundation of optimal control, VisioPath represents a significant step forward in safe trajectory planning for complex traffic systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Trajectory PlanningAutonomous DrivingSimilar Papers 제목 키워드 기반
Enhanced Frequency Containment Reserve Provision from Battery Hybridized Hydropower Plants: Theory and Experimental Validation
This paper presents a solution to address wear and tear of Run-of-River (RoR) Hydropower Plants (HPPs) providing enhanced Frequency Containment Reserve (FCR). In this respect, the study proposes the integration of a Batt…
Model Predictive ControlOn the Practical Design of Tube-Enhanced Multi-Stage Nonlinear Model Predictive Control
Tube-enhanced multi-stage nonlinear model predictive control is a robust control scheme that can handle a wide range of uncertainties with reduced conservatism and manageable computational complexity. In this paper, we e…
Model Predictive ControlA Tutorial on Gaussian Process Learning-based Model Predictive Control
This tutorial provides a systematic introduction to Gaussian process learning-based model predictive control (GP-MPC), an advanced approach integrating Gaussian process (GP) with model predictive control (MPC) for enhanc…
Model Predictive ControlExperimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
Accurate occupancy information is essential for closed-loop occupant-centric control (OCC) in smart buildings. However, existing vision-based occupancy measurement methods often struggle to provide stable and accurate me…
Toward Context-Aware Exoskeleton Assistance: Integrating Computer Vision Payload Estimation with a Multi-Metric Optimization Space
Back-support exoskeletons mitigate musculoskeletal strain, yet current systems rely on reactive sensing and lack context-aware assistance modulation. This paper presents a population-derived optimization framework and a …