Driving Video Retrieval for Complex Queries with Structured Grounding
Video retrieval at scale is central to data curation and safety validation in autonomous driving, where users want to find not only scenes but also dynamic events such as cut-ins and hard braking. Existing vision-language and keyword-based retrieval methods often miss these events because the relevant motion may not be explicitly described in text or captured by lexical overlap. Rule-based retrieval can encode such events more directly, but it is brittle: generated or hand-written rules often fail when their assumptions do not match real driving data. We propose STRIVE-D, a data-calibrated retrieval framework for driving videos. It uses weakly labeled in-domain videos to estimate when a query rule is reliable, adapt rules that mismatch observed data, and fuse calibrated rule scores with vision-language and keyword-based retrieval signals. Across three driving benchmarks, including newly released human-annotated event data on DrivingDojo, STRIVE-D delivers up to 84% relative improvement in top-1 accuracy over state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingVideo RetrievalSimilar Papers 제목 키워드 기반
Visual Semantic Search: Retrieving Videos via Complex Textual Queries
In this paper, we tackle the problem of retrieving videos using complex natural language queries. Towards this goal, we first parse the sentential descriptions into a semantic graph, which is then matched to visual conce…
Autonomous DrivingNatural Language QueriesRetrievalVideo RetrievalDialogue-to-Video Retrieval
Recent years have witnessed an increasing amount of dialogue/conversation on the web especially on social media. That inspires the development of dialogue-based retrieval, in which retrieving videos based on dialogue is …
Recommendation SystemsRetrievalVideo RetrievalTree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval
The rapid growth of user-generated videos on the Internet has intensified the need for text-based video retrieval systems. Traditional methods mainly favor the concept-based paradigm on retrieval with simple queries, whi…
RetrievalVideo RetrievalInduce, Edit, Retrieve:Language Grounded Multimodal Schema for Instructional Video Retrieval
Schemata are structured representations of complex tasks that can aid artificial intelligence by allowing models to break down complex tasks into intermediate steps. We propose a novel system that induces schemata from w…
RetrievalVideo RetrievalSTaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Bases
Answering real-world complex queries, such as complex product search, often requires accurate retrieval from semi-structured knowledge bases that involve blend of unstructured (e.g., textual descriptions of products) and…
BenchmarkingRetrieval