GenState-AI: State-Aware Dataset for Text-to-Video Retrieval on AI-Generated Videos
Existing text-to-video retrieval benchmarks are dominated by real-world footage where much of the semantics can be inferred from a single frame, leaving temporal reasoning and explicit end-state grounding under-evaluated. We introduce GenState-AI, an AI-generated benchmark centered on controlled state transitions, where each query is paired with a main video, a temporal hard negative that differs only in the decisive end-state, and a semantic hard negative with content substitution, enabling fine-grained diagnosis of temporal vs. semantic confusions beyond appearance matching. Using Wan2.2-TI2V-5B, we generate short clips whose meaning depends on precise changes in position, quantity, and object relations, providing controllable evaluation conditions for state-aware retrieval. We evaluate two representative MLLM-based baselines, and observe consistent and interpretable failure patterns: both frequently confuse the main video with the temporal hard negative and over-prefer temporally plausible but end-state-incorrect clips, indicating insufficient grounding to decisive end-state evidence, while being comparatively less sensitive to semantic substitutions. We further introduce triplet-based diagnostic analyses, including relative-order statistics and breakdowns across transition categories, to make temporal vs. semantic failure sources explicit. GenState-AI provides a focused testbed for state-aware, temporally and semantically sensitive text-to-video retrieval, and will be released on huggingface.co.
Code (0)
등록된 구현이 없습니다.
Tasks
Video RetrievalSimilar Papers 제목 키워드 기반
Principal eigenstate classical shadows
Given many copies of an unknown quantum state $\rho$, we consider the task of learning a classical description of its principal eigenstate. Namely, assuming that $\rho$ has an eigenstate $|\phi\rangle$ with (unknown) eig…
A smooth basis for atomistic machine learning
Machine learning frameworks based on correlations of interatomic positions begin with a discretized description of the density of other atoms in the neighbourhood of each atom in the system. Symmetry considerations suppo…
Physics-Informed Neural Networks for Discovering Localised Eigenstates in Disordered Media
The Schr\"{o}dinger equation with random potentials is a fundamental model for understanding the behaviour of particles in disordered systems. Disordered media are characterised by complex potentials that lead to the loc…
Equilibrium Propagation: the Quantum and the Thermal Cases
Equilibrium propagation is a recently introduced method to use and train artificial neural networks in which the network is at the minimum (more generally extremum) of an energy functional. Equilibrium propagation has sh…
Transactional Interpretation for the Principle of Minimum Fisher Information
The principle of minimum Fisher information states that in the set of acceptable probability distributions characterizing the given system, it is best done by the one that minimizes the corresponding Fisher information. …