Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations
Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view, etc. We present a framework to visualize the missed regions per-frame during the colonoscopy, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC domain. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared latent space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. This additional noise input can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.
Code (1)
Tasks
Image-to-Image TranslationTranslationSimilar Papers 제목 키워드 기반
Estimating the coverage in 3d reconstructions of the colon from colonoscopy videos
Colonoscopy is the most common procedure for early detection and removal of polyps, a critical component of colorectal cancer prevention. Insufficient visual coverage of the colon surface during the procedure often resul…
ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptation
Visualizing colonoscopy is crucial for medical auxiliary diagnosis to prevent undetected polyps in areas that are not fully observed. Traditional feature-based and depth-based reconstruction approaches usually end up wit…
Depth EstimationDomain AdaptationLighting Enhancement Aids Reconstruction of Colonoscopic Surfaces
High screening coverage during colonoscopy is crucial to effectively prevent colon cancer. Previous work has allowed alerting the doctor to unsurveyed regions by reconstructing the 3D colonoscopic surface from colonoscop…
Surface ReconstructionDeep Learning-based Biological Anatomical Landmark Detection in Colonoscopy Videos
Colonoscopy is a standard imaging tool for visualizing the entire gastrointestinal (GI) tract of patients to capture lesion areas. However, it takes the clinicians excessive time to review a large number of images extrac…
Anatomical Landmark DetectionDeep LearningC$^3$Fusion: Consistent Contrastive Colon Fusion, Towards Deep SLAM in Colonoscopy
3D colon reconstruction from Optical Colonoscopy (OC) to detect non-examined surfaces remains an unsolved problem. The challenges arise from the nature of optical colonoscopy data, characterized by highly reflective low-…
Pose Estimation