Probing Language Models for Understanding of Temporal Expressions
We present three Natural Language Inference (NLI) challenge sets that can evaluate NLI models on their understanding of temporal expressions. More specifically, we probe these models for three temporal properties: (a) the order between points in time, (b) the duration between two points in time, (c) the relation between the magnitude of times specified in different units. We find that although large language models fine-tuned on MNLI have some basic perception of the order between points in time, at large, these models do not have a thorough understanding of the relation between temporal expressions.
Code (1)
Tasks
Natural Language InferenceProbing Language ModelsRelationSimilar Papers 제목 키워드 기반
HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning
Understanding long videos requires fine-grained perception and multi-step, higher-order reasoning over complex, long-range spatio-temporal dynamics. Vision-language models (VLMs) encode video frames into visual tokens an…
Relational ReasoningSemantic RetrievalDynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views
Temporal concept drift refers to the problem of data changing over time. In NLP, that would entail that language (e.g. new expressions, meaning shifts) and factual knowledge (e.g. new concepts, updated facts) evolve over…
BenchmarkingCogCompTime: A Tool for Understanding Time in Natural Language Text
Automatic extraction of temporal information in text is an important component of natural language understanding. It involves two basic tasks: (1) Understanding time expressions that are mentioned explicitly in text (e.g…
Natural Language UnderstandingCogCompTime: A Tool for Understanding Time in Natural Language
Automatic extraction of temporal information is important for natural language understanding. It involves two basic tasks: (1) Understanding time expressions that are mentioned explicitly in text (e.g., February 27, 1998…
Natural Language UnderstandingQuestion AnsweringSMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models
Despite the recent advances of the artificial intelligence, building social intelligence remains a challenge. Among social signals, laughter is one of the distinctive expressions that occurs during social interactions be…
Video Understanding