CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards this direction, we introduce CRAFT, a new video question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 58K video and question pairs that are generated from 10K videos from 20 different virtual environments, containing various objects in motion that interact with each other and the scene. Two question categories in CRAFT include previously studied descriptive and counterfactual questions. Additionally, inspired by the Force Dynamics Theory in cognitive linguistics, we introduce a new causal question category that involves understanding the causal interactions between objects through notions like cause, enable, and prevent. Our results show that even though the questions in CRAFT are easy for humans, the tested baseline models, including existing state-of-the-art methods, do not yet deal with the challenges posed in our benchmark.
Code (1)
Tasks
counterfactualDescriptiveQuestion AnsweringVideo Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards …
counterfactualDescriptiveQuestion AnsweringVideo Question AnsweringThink before You Simulate: Symbolic Reasoning to Orchestrate Neural Computation for Counterfactual Question Answering
Causal and temporal reasoning about video dynamics is a challenging problem. While neuro-symbolic models that combine symbolic reasoning with neural-based perception and prediction have shown promise, they exhibit limita…
counterfactualCounterfactual ReasoningQuestion AnsweringCausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos
Recent advances in large language models (LLMs) have improved reasoning in text and image domains, yet achieving robust video reasoning remains a significant challenge. Existing video benchmarks mainly assess shallow und…
CausalProfiler: Generating Synthetic Benchmarks for Rigorous and Transparent Evaluation of Causal Machine Learning
Causal machine learning (Causal ML) aims to answer "what if" questions using machine learning algorithms, making it a promising tool for high-stakes decision-making. Yet, empirical evaluation practices in Causal ML remai…
CausalAffect: Causal Discovery for Facial Affective Understanding
Understanding human affect from facial behavior requires not only accurate recognition but also structured reasoning over the latent dependencies that drive muscle activations and their expressive outcomes. Although Acti…