Papers Spatio-Temporal Video Grounding
“Spatio-Temporal Video Grounding” 태그가 달린 논문 37편 · 필터 해제
Learning Compositional Spatio-Temporal Video Grounding with Synthetic Curriculum
Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on simple queries. They largely overlook the compositional queries prevale…
Spatio-Temporal Video GroundingReinforcement LearningLocate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding
Spatio-temporal video grounding (STVG) requires models to identify when a referred event occurs and localize the target entity throughout that interval. Existing multimodal large language models typically serialize dense…
Spatio-Temporal Video GroundingVideo Object TrackingAnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models
Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchm…
Spatio-Temporal Video GroundingZero-shot GeneralizationDomain AdaptationKnowledge-Preserved Model Tuning in Null-Space for Robust Spatio-Temporal Video Grounding
Spatio-Temporal Video Grounding aims to localize object tubes based on textual queries. While recent methods have achieved remarkable success, they mainly focus on high-quality(HQ) inputs, neglecting the widespread prese…
Spatio-Temporal Video GroundingUnlocking the Potential of Grounding DINO in Videos: Parameter-Efficient Adaptation for Limited-Data Spatial-Temporal Localization
Spatio-temporal video grounding (STVG) aims to localize queried objects within dynamic video segments. Prevailing fully-trained approaches are notoriously data-hungry. However, gathering large-scale STVG data is exceptio…
Spatio-Temporal Video GroundingBridging Time and Space: Decoupled Spatio-Temporal Alignment for Video Grounding
Spatio-Temporal Video Grounding requires jointly localizing target objects across both temporal and spatial dimensions based on natural language queries, posing fundamental challenges for existing Multimodal Large Langua…
Spatio-Temporal Video GroundingNatural Language QueriesTowards Long-Form Spatio-Temporal Video Grounding
In real scenarios, videos can span several minutes or even hours. However, existing research on spatio-temporal video grounding (STVG), given a textual query, mainly focuses on localizing targets in short videos of tens …
Spatio-Temporal Video GroundingAgentic Spatio-Temporal Grounding via Collaborative Reasoning
Spatio-Temporal Video Grounding (STVG) aims to retrieve the spatio-temporal tube of a target object or person in a video given a text query. Most existing approaches perform frame-wise spatial localization within a predi…
Spatio-Temporal Video GroundingSpatial ReasoningDetector-Empowered Video Large Language Model for Efficient Spatio-Temporal Grounding
Multimodal large language models (MLLMs) are rapidly expanding from general video understanding to finer-grained understanding such as spatio-temporal video grounding (STVG) and reasoning. In these tasks, an MLLM must lo…
Spatio-Temporal Video GroundingToG-Bench: Task-Oriented Spatio-Temporal Grounding in Egocentric Videos
A core capability towards general embodied intelligence lies in localizing task-relevant objects from an egocentric perspective, formulated as Spatio-Temporal Video Grounding (STVG). Despite recent progress, existing STV…
Spatio-Temporal Video GroundingThinking With Bounding Boxes: Enhancing Spatio-Temporal Video Grounding via Reinforcement Fine-Tuning
Spatio-temporal video grounding (STVG) requires localizing a target object in untrimmed videos both temporally and spatially from natural language descriptions. Despite their strong language understanding, multimodal lar…
Spatio-Temporal Video GroundingWord AlignmentOmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios
Spatio-Temporal Video Grounding (STVG) aims to localize target objects in videos based on natural language descriptions. Despite recent advances in Multimodal Large Language Models, a significant gap remains between curr…
Spatio-Temporal Video GroundingTubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding
Spatio-Temporal Video Grounding (STVG) aims to localize a spatio-temporal tube that corresponds to a given language query in an untrimmed video. This is a challenging task since it involves complex vision-language unders…
Spatio-Temporal Video GroundingVisual GroundingImage-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey
Image-Language Foundation Models (ILFMs) have demonstrated remarkable success in vision-language understanding, providing transferable multimodal representations that generalize across diverse downstream image-based task…
Spatio-Temporal Video GroundingVideo Question AnsweringTransfer LearningUnleashing the Potential of Multimodal LLMs for Zero-Shot Spatio-Temporal Video Grounding
Spatio-temporal video grounding (STVG) aims at localizing the spatio-temporal tube of a video, as specified by the input text query. In this paper, we utilize multimodal large language models (MLLMs) to explore a zero-sh…
Spatio-Temporal Video GroundingSpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability
Multimodal large language models (MLLMs) have made remarkable progress in either temporal or spatial localization. However, they struggle to perform spatio-temporal video grounding. This limitation stems from two major c…
Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language Model+3Large-scale Pre-training for Grounded Video Caption Generation
We propose a novel approach for captioning and object grounding in video, where the objects in the caption are grounded in the video via temporally dense bounding boxes. We introduce the following contributions. First, w…
Caption GenerationSpatio-Temporal Video GroundingKnowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding
Transformer has attracted increasing interest in STVG, owing to its end-to-end pipeline and promising result. Existing Transformer-based STVG approaches often leverage a set of object queries, which are initialized simpl…
AttributeObjectSpatio-Temporal Video GroundingTemporal Localization+1Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Grounding
In this work, we focus on Weakly Supervised Spatio-Temporal Video Grounding (WSTVG). It is a multimodal task aimed at localizing specific subjects spatio-temporally based on textual queries without bounding box supervisi…
object-detectionObject DetectionPhrase GroundingScene Understanding+2STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding
In this work, we study Weakly Supervised Spatio-Temporal Video Grounding (WSTVG), a challenging task of localizing subjects spatio-temporally in videos using only textual queries and no bounding box supervision. Insp…
Action UnderstandingSpatio-Temporal Video GroundingVideo Grounding