LAVA: Language Driven Scalable and Versatile Traffic Video Analytics
In modern urban environments, camera networks generate massive amounts of operational footage -- reaching petabytes each day -- making scalable video analytics essential for efficient processing. Many existing approaches adopt an SQL-based paradigm for querying such large-scale video databases; however, this constrains queries to rigid patterns with predefined semantic categories, significantly limiting analytical flexibility. In this work, we explore a language-driven video analytics paradigm aimed at enabling flexible and efficient querying of high-volume video data driven by natural language. Particularly, we build \textsc{Lava}, a system that accepts natural language queries and retrieves traffic targets across multiple levels of granularity and arbitrary categories. \textsc{Lava} comprises three main components: 1) a multi-armed bandit-based efficient sampling method for video segment-level localization; 2) a video-specific open-world detection module for object-level retrieval; and 3) a long-term object trajectory extraction scheme for temporal object association, yielding complete trajectories for object-of-interests. To support comprehensive evaluation, we further develop a novel benchmark by providing diverse, semantically rich natural language predicates and fine-grained annotations for multiple videos. Experiments on this benchmark demonstrate that \textsc{Lava} improves $F_1$-scores for selection queries by $\mathbf{14\%}$, reduces MPAE for aggregation queries by $\mathbf{0.39}$, and achieves top-$k$ precision of $\mathbf{86\%}$, while processing videos $ \mathbf{9.6\times} $ faster than the most accurate baseline. Our code and dataset are available at https://github.com/yuyanrui/LAVA.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language QueriesSimilar Papers 제목 키워드 기반
Scenario Understanding of Traffic Scenes Through Large Visual Language Models
Deep learning models for autonomous driving, encompassing perception, planning, and control, depend on vast datasets to achieve their high performance. However, their generalization often suffers due to domain-specific d…
Autonomous DrivingMVLLaVA: An Intelligent Agent for Unified and Flexible Novel View Synthesis
This paper introduces MVLLaVA, an intelligent agent designed for novel view synthesis tasks. MVLLaVA integrates multiple multi-view diffusion models with a large multimodal model, LLaVA, enabling it to handle a wide rang…
DescriptiveNovel View SynthesisLeveraging Multimodal-LLMs Assisted by Instance Segmentation for Intelligent Traffic Monitoring
A robust and efficient traffic monitoring system is essential for smart cities and Intelligent Transportation Systems (ITS), using sensors and cameras to track vehicle movements, optimize traffic flow, reduce congestion,…
Instance SegmentationLanguage ModelingLanguage ModellingLarge Language Model+3Scalable Adaptive Traffic Light Control Over a Traffic Network Including Turns, Transit Delays, and Blocking
We develop adaptive data-driven traffic light controllers for a grid-like traffic network considering straight, left-turn, and right-turn traffic flows. The analysis incorporates transit delays and blocking effects on ve…
BlockingScalable Adaptive Traffic Light Control Over a Traffic Network Including Transit Delays
We study the Traffic Light Control (TLC) problem for a traffic network with multiple intersections in an artery, including the effect of transit delays for vehicles moving from one intersection to the next. The goal is t…