Dynamic Fluid Surface Reconstruction Using Deep Neural Network
Recovering the dynamic fluid surface is a long-standing challenging problem in computer vision. Most existing image-based methods require multiple views or a dedicated imaging system. Here we present a learning-based single-image approach for 3D fluid surface reconstruction. Specifically, we design a deep neural network that estimates the depth and normal maps of a fluid surface by analyzing the refractive distortion of a reference background image. Due to the dynamic nature of fluid surfaces, our network uses recurrent layers that carry temporal information from previous frames to achieve spatio-temporally consistent reconstruction given a video input. Due to the lack of fluid data, we synthesize a large fluid dataset using physics-based fluid modeling and rendering techniques for network training and validation. Through experiments on simulated and real captured fluid images, we demonstrate that our proposed deep neural network trained on our fluid dataset can recover dynamic 3D fluid surfaces with high accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Surface ReconstructionSimilar Papers 제목 키워드 기반
Stereo-Based 3D Reconstruction of Dynamic Fluid Surfaces by Global Optimization
3D reconstruction of dynamic fluid surfaces is an open and challenging problem in computer vision. Unlike previous approaches that reconstruct each surface point independently and often return noisy depth maps, we propos…
3D geometry3D Reconstructionglobal-optimizationFluidGaussian: Propagating Simulation-Based Uncertainty Toward Functionally-Intelligent 3D Reconstruction
Real objects that inhabit the physical world follow physical laws and thus behave plausibly during interaction with other physical objects. However, current methods that perform 3D reconstructions of real-world scenes fr…
3D ReconstructionActive LearningA GPU-based Hydrodynamic Simulator with Boid Interactions
We present a hydrodynamic simulation system using the GPU compute shaders of DirectX for simulating virtual agent behaviors and navigation inside a smoothed particle hydrodynamical (SPH) fluid environment with real-time …
GPUSurface ReconstructionFluid Dynamics Network: Topology-Agnostic 4D Reconstruction via Fluid Dynamics Priors
Representing 3D surfaces as level sets of continuous functions over $\mathbb{R}^3$ is the common denominator of neural implicit representations, which recently enabled remarkable progress in geometric deep learning and c…
4D reconstructionNeReF: Neural Refractive Field for Fluid Surface Reconstruction and Implicit Representation
Existing neural reconstruction schemes such as Neural Radiance Field (NeRF) are largely focused on modeling opaque objects. We present a novel neural refractive field(NeReF) to recover wavefront of transparent fluids by …
global-optimizationNeRFSurface Reconstruction