ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors
Three-dimensional (3D) scene understanding in colonoscopy presents significant challenges that necessitate automated methods for accurate depth estimation. However, existing depth estimation models for endoscopy struggle with temporal consistency across video sequences, limiting their applicability for 3D reconstruction. We present ColonCrafter, a diffusion-based depth estimation model that generates temporally consistent depth maps from monocular colonoscopy videos. Our approach learns robust geometric priors from synthetic colonoscopy sequences to generate temporally consistent depth maps. We also introduce a style transfer technique that preserves geometric structure while adapting real clinical videos to match our synthetic training domain. ColonCrafter achieves state-of-the-art zero-shot performance on the C3VD dataset, outperforming both general-purpose and endoscopy-specific approaches. Although full trajectory 3D reconstruction remains a challenge, we demonstrate clinically relevant applications of ColonCrafter, including 3D point cloud generation and surface coverage assessment.
Code (0)
등록된 구현이 없습니다.
Tasks
Point Cloud GenerationScene Understanding3D ReconstructionDepth EstimationSimilar Papers 제목 키워드 기반
ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptation
Visualizing colonoscopy is crucial for medical auxiliary diagnosis to prevent undetected polyps in areas that are not fully observed. Traditional feature-based and depth-based reconstruction approaches usually end up wit…
Depth EstimationDomain AdaptationLeveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images
Colonoscopy is the choice procedure to diagnose colon and rectum cancer, from early detection of small precancerous lesions (polyps), to confirmation of malign masses. However, the high variability of the organ appearanc…
Depth EstimationSoftEnNet: Symbiotic Monocular Depth Estimation and Lumen Segmentation for Colonoscopy Endorobots
Colorectal cancer is the third most common cause of cancer death worldwide. Optical colonoscopy is the gold standard for detecting colorectal cancer; however, about 25 percent of polyps are missed during the procedure. A…
Depth EstimationMonocular Depth EstimationRobot NavigationSegmentationColonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration
Screening colonoscopy is an important clinical application for several 3D computer vision techniques, including depth estimation, surface reconstruction, and missing region detection. However, the development, evaluation…
BenchmarkingDepth EstimationGenerative Adversarial NetworkOptical Flow Estimation+2ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction
One of the key elements of reconstructing a 3D mesh from a monocular video is generating every frame's depth map. However, in the application of colonoscopy video reconstruction, producing good-quality depth estimation i…
3D ReconstructionDepth EstimationVideo Reconstruction