Burn After Reading: Do Multimodal Large Language Models Truly Capture Order of Events in Image Sequences?
This paper introduces the TempVS benchmark, which focuses on temporal grounding and reasoning capabilities of Multimodal Large Language Models (MLLMs) in image sequences. TempVS consists of three main tests (i.e., event relation inference, sentence ordering and image ordering), each accompanied with a basic grounding test. TempVS requires MLLMs to rely on both visual and linguistic modalities to understand the temporal order of events. We evaluate 38 state-of-the-art MLLMs, demonstrating that models struggle to solve TempVS, with a substantial performance gap compared to human capabilities. We also provide fine-grained insights that suggest promising directions for future research. Our TempVS benchmark data and code are available at https://github.com/yjsong22/TempVS.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence OrderingSimilar Papers 제목 키워드 기반
Burn After Reading: Online Adaptation for Cross-domain Streaming Data
In the context of online privacy, many methods propose complex privacy and security preserving measures to protect sensitive data. In this paper, we argue that: not storing any sensitive data is the best form of security…
DiversityDomain AdaptationUnsupervised Domain AdaptationMultimodal Noisy Segmentation based fragmented burn scars identification in Amazon Rainforest
Detection of burn marks due to wildfires in inaccessible rain forests is important for various disaster management and ecological studies. The fragmented nature of arable landscapes and diverse cropping patterns often th…
DecoderManagementHuman Activity Recognition using Smartphones
Human Activity Recognition is a subject of great research today and has its applications in remote healthcare, activity tracking of the elderly or the disables, calories burnt tracking etc. In our project, we have create…
Activity RecognitionDimensionality ReductionHuman Activity RecognitionMathematical Modeling, In-Human Evaluation and Analysis of Volume Kinetics and Kidney Function after Burn Injury and Resuscitation
Existing burn resuscitation protocols exhibit large variability in treatment efficacy. Hence, they must be further optimized based on comprehensive knowledge of burn pathophysiology. A physics-based mathematical model th…
Kidney FunctionAI-based approach to burnout identification from textual data
This study introduces an AI-based methodology that utilizes natural language processing (NLP) to detect burnout from textual data. The approach relies on a RuBERT model originally trained for sentiment analysis and subse…
Sentiment Analysis