RGB to Hyperspectral: Spectral Reconstruction for Enhanced Surgical Imaging
This study investigates the reconstruction of hyperspectral signatures from RGB data to enhance surgical imaging, utilizing the publicly available HeiPorSPECTRAL dataset from porcine surgery and an in-house neurosurgery dataset. Various architectures based on convolutional neural networks (CNNs) and transformer models are evaluated using comprehensive metrics. Transformer models exhibit superior performance in terms of RMSE, SAM, PSNR and SSIM by effectively integrating spatial information to predict accurate spectral profiles, encompassing both visible and extended spectral ranges. Qualitative assessments demonstrate the capability to predict spectral profiles critical for informed surgical decision-making during procedures. Challenges associated with capturing both the visible and extended hyperspectral ranges are highlighted using the MAE, emphasizing the complexities involved. The findings open up the new research direction of hyperspectral reconstruction for surgical applications and clinical use cases in real-time surgical environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingSpectral ReconstructionSSIMMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HyKey: Hyperspectral Keypoint Detection and Matching in Minimally Invasive Surgery
Purpose: 3D reconstruction in minimally invasive surgery (MIS) enables enhanced surgical guidance through improved visualisation, tool tracking, and augmented reality. However, traditional RGB-based keypoint detection an…
Keypoint Detection3D ReconstructionPose EstimationA self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging
Hyperspectral imaging holds promises in surgical imaging by offering biological tissue differentiation capabilities with detailed information that is invisible to the naked eye. For intra-operative guidance, real-time sp…
DemosaickingDeep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction
Hyperspectral imaging is one of the most promising techniques for intraoperative tissue characterisation. Snapshot mosaic cameras, which can capture hyperspectral data in a single exposure, have the potential to make a r…
Decision MakingDemosaickingHyperspectral Image Super-ResolutionImage Generation+3Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging
Intra-operative measurements of tissue shape and multi/ hyperspectral information have the potential to provide surgical guidance and decision making support. We report an optical probe based system to combine sparse hyp…
Decision MakingOptical Flow EstimationSynthetic white balancing for intra-operative hyperspectral imaging
Hyperspectral imaging shows promise for surgical applications to non-invasively provide spatially-resolved, spectral information. For calibration purposes, a white reference image of a highly-reflective Lambertian surfac…