HyperColorization: Propagating spatially sparse noisy spectral clues over hyperspectral images

1: Northwestern University Computer Science
2: Purdue University Electrical and Computer Engineering

Our framework reconstructs a dense hyperspectral image from a grayscale guide image and spatially sparse noisy spectral clues with the simple premise that pixels with similar monochromatic illuminance should have similar spectral reflectance.


Hyperspectral cameras face challenging spatial-spectral resolution trade-offs and are more affected by shot noise than RGB photos taken over the same total exposure time. Here, we present a colorization algorithm to reconstruct hyperspectral images from a grayscale guide image and spatially sparse spectral clues. We demonstrate that our algorithm generalizes to lower dimensional models for hyperspectral images, and show that colorizing in a low-rank space reduces compute time and the impact of shot noise. To enhance robustness, we incorporate guided sampling, edge-aware filtering, and dimensionality estimation techniques. Our method surpasses previous algorithms in various performance metrics, including SSIM, PSNR, GFC, and EMD, which we propose as valuable metrics for characterizing hyperspectral image quality. Collectively, these findings provide a promising avenue for overcoming the time-space-wavelength resolution trade-off by reconstructing a dense hyperspectral image from samples obtained by whisk or push broom scanners, as well as computational imaging systems.



        author = {M. Kerem Aydin and Qi Guo and Emma Alexander},
        journal = {Opt. Express},
        keywords = {Computational imaging; Hyperspectral imaging; Image quality; Imaging systems; Shot noise; Spatial resolution},
        number = {7},
        pages = {10761--10776},
        publisher = {Optica Publishing Group},
        title = {HyperColorization: propagating spatially sparse noisy spectral clues for reconstructing hyperspectral images},
        volume = {32},
        month = {Mar},
        year = {2024},
        url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-32-7-10761},
        doi = {10.1364/OE.508017},