paper-with-me

홈 › Papers

Modulate and Reconstruct: Learning Hyperspectral Imaging from Misaligned Smartphone Views

2025-07-02 · Daniil Reutsky, Daniil Vladimirov, Yasin Mamedov, Georgy Perevozchikov, Nancy Mehta, Egor Ershov, Radu Timofte arxiv

Hyperspectral reconstruction (HSR) from RGB images is a highly promising direction for accurate color reproduction and material color measurement. While most existing approaches rely on a single RGB image - thereby limiting reconstruction accuracy - the majority of modern smartphones are equipped with two or more cameras. In this work, we propose a novel multi-image-to-hyperspectral reconstruction (MI-HSR) framework that leverages a triple-camera smartphone system, where two lenses are equipped with carefully selected spectral filters. Our easy-to-implement configuration, based on theoretical and empirical analysis, allows to obtain more complete and diverse spectral data than traditional single-chamber setups. To support this new paradigm, we introduce Doomer, the first dataset for MI-HSR, comprising aligned images from three smartphone cameras and a hyperspectral reference camera across diverse scenes. We further introduce a lightweight alignment module for MI-HSR that effectively fuses multi-view inputs while mitigating parallax- and occlusion-induced artifacts. Proposed module demonstrate consistent quality improvements for modern HSR methods. In a nutshell, our setup allows 30% more accurate estimations of spectra compared to an ordinary RGB camera, while the proposed alignment module boosts the reconstruction quality of SotA methods by an additional 5%. Our findings suggest that spectral filtering of multiple views with commodity hardware unlocks more accurate and practical hyperspectral imaging.

📄 PDF Abstract BibTeX arXiv:2507.01835

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Hyperspectral image reconstruction by deep learning with super-Rayleigh speckles

2025-02-26 · Ziyan Chen, Zhentao Liu, Jianrong Wu, Shensheng Han

Ghost imaging via sparsity constraints (GISC) spectral camera modulates the three-dimensional (3D) hyperspectral image into a two-dimensional (2D) compressive image with speckles in a single shot. It obtains a 3D hypersp…

Deep LearningImage Reconstruction

mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

2023-03-27 · Yuhyun Ji, Sang Mok Park, Semin Kwon, Jung Woo Leem 외

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instrumen…

compressed sensing

Robust Image Reconstruction with Misaligned Structural Information

2020-04-01 · Leon Bungert, Matthias J. Ehrhardt

Multi-modality (or multi-channel) imaging is becoming increasingly important and more widely available, e.g. hyperspectral imaging in remote sensing, spectral CT in material sciences as well as multi-contrast MRI and PET…

Image Reconstruction

FUN: A Focal U-Net Combining Reconstruction and Object Detection for Snapshot Spectral Imaging

2026-04-30 · Dahua Gao, Yubo Dong, Anqi Li, Zhenyuan Lin 외 arxiv

Conventional push-broom hyperspectral imaging suffers from slow acquisition speeds, precluding real-time object detection; in contrast, snapshot spectral imaging enables instantaneous hyperspectral images (HSIs) capture,…

Real-Time Object DetectionMulti-Task Learning

Proximal Gradient Descent Unfolding Dense-spatial Spectral-attention Transformer for Compressive Spectral Imaging

2023-12-25 · Ziyan Chen, Jing Cheng

The Coded Aperture Snapshot Spectral Compressive Imaging (CASSI) system modulates three-dimensional hyperspectral images into two-dimensional compressed images in a single exposure. Subsequently, three-dimensional hypers…

GPU