ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learning
We introduce \textit{ImmersePro}, an innovative framework specifically designed to transform single-view videos into stereo videos. This framework utilizes a novel dual-branch architecture comprising a disparity branch and a context branch on video data by leveraging spatial-temporal attention mechanisms. \textit{ImmersePro} employs implicit disparity guidance, enabling the generation of stereo pairs from video sequences without the need for explicit disparity maps, thus reducing potential errors associated with disparity estimation models. In addition to the technical advancements, we introduce the YouTube-SBS dataset, a comprehensive collection of 423 stereo videos sourced from YouTube. This dataset is unprecedented in its scale, featuring over 7 million stereo pairs, and is designed to facilitate training and benchmarking of stereo video generation models. Our experiments demonstrate the effectiveness of \textit{ImmersePro} in producing high-quality stereo videos, offering significant improvements over existing methods. Compared to the best competitor stereo-from-mono we quantitatively improve the results by 11.76\% (L1), 6.39\% (SSIM), and 5.10\% (PSNR).
Code (1)
Tasks
BenchmarkingDisparity EstimationSSIMVideo GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Neural Stereo Video Compression with Hybrid Disparity Compensation
Disparity compensation represents the primary strategy in stereo video compression (SVC) for exploiting cross-view redundancy. These mechanisms can be broadly categorized into two types: one that employs explicit horizon…
Autonomous DrivingVideo CompressionStereo World Model: Camera-Guided Stereo Video Generation
We present StereoWorld, a camera-conditioned stereo world model that jointly learns appearance and binocular geometry for end-to-end stereo video generation.Unlike monocular RGB or RGBD approaches, StereoWorld operates e…
Depth EstimationVideo GenerationEye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesis
The rising popularity of immersive visual experiences has increased interest in stereoscopic 3D video generation. Despite significant advances in video synthesis, creating 3D videos remains challenging due to the relativ…
Disparity EstimationTransparent objectsVideo GenerationStereo-Knowledge Distillation from dpMV to Dual Pixels for Light Field Video Reconstruction
Dual pixels contain disparity cues arising from the defocus blur. This disparity information is useful for many vision tasks ranging from autonomous driving to 3D creative realism. However, directly estimating disparity …
Autonomous DrivingKnowledge DistillationVideo ReconstructionSpatialMe: Stereo Video Conversion Using Depth-Warping and Blend-Inpainting
Stereo video conversion aims to transform monocular videos into immersive stereo format. Despite the advancements in novel view synthesis, it still remains two major challenges: i) difficulty of achieving high-fidelity a…
Novel View Synthesis