Do generative video models understand physical principles?
AI video generation is undergoing a revolution, with quality and realism advancing rapidly. These advances have led to a passionate scientific debate: Do video models learn "world models" that discover laws of physics -- or, alternatively, are they merely sophisticated pixel predictors that achieve visual realism without understanding the physical principles of reality? We address this question by developing Physics-IQ, a comprehensive benchmark dataset that can only be solved by acquiring a deep understanding of various physical principles, like fluid dynamics, optics, solid mechanics, magnetism and thermodynamics. We find that across a range of current models (Sora, Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet), physical understanding is severely limited, and unrelated to visual realism. At the same time, some test cases can already be successfully solved. This indicates that acquiring certain physical principles from observation alone may be possible, but significant challenges remain. While we expect rapid advances ahead, our work demonstrates that visual realism does not imply physical understanding. Our project page is at https://physics-iq.github.io; code at https://github.com/google-deepmind/physics-IQ-benchmark.
Code (1)
Tasks
Video GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GenMatter: Perceiving Physical Objects with Generative Matter Models
Human visual perception offers valuable insights for understanding computational principles of motion-based scene interpretation. Humans robustly detect and segment moving entities that constitute independently moveable …
Object SegmentationScene UnderstandingPhysVid: Physics Aware Local Conditioning for Generative Video Models
Generative video models achieve high visual fidelity but often violate basic physical principles, limiting reliability in real-world settings. Prior attempts to inject physics rely on conditioning: frame-level signals ar…
Physical Informed Driving World Model
Autonomous driving requires robust perception models trained on high-quality, large-scale multi-view driving videos for tasks like 3D object detection, segmentation and trajectory prediction. While world models provide a…
3D Object DetectionAutonomous Drivingmodelobject-detection+3PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Video-language models (VLMs) have achieved remarkable performance on video understanding and visual question answering, yet they remain unreliable in reasoning about physical plausibility, where understanding object inte…
Physical Commonsense ReasoningVisual Question AnsweringT2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation
Text-to-video generative models have made significant strides in recent years, producing high-quality videos that excel in both aesthetic appeal and accurate instruction following, and have become central to digital art …
counterfactualInstruction FollowingText-to-Video GenerationVideo Generation