Abstract
Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modeling, and medical imaging. It is nontrivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with a deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regular-izer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using a deep learning denoiser and a plug-and-play model.
Original language | English |
---|---|
Pages (from-to) | 1177-1194 |
Number of pages | 18 |
Journal | SIAM Journal on Imaging Sciences |
Volume | 16 |
Issue number | 3 |
DOIs | |
Publication status | Published - 2023 |
Externally published | Yes |
Bibliographical note
Publisher Copyright:© 2023 Society for Industrial and Applied Mathematics.
Funding
This work was supported in part by the National Key R\&D Program of China under grant 2021YFE0203700, grant NSFC/RGC N CUHK 415/19, NSFC grants 11871210, 11971215, and 61971292, grant ITF MHP/038/20, RGC grants 14300219, 14302920, and 14301121 and a CUHK Direct Grant for Research. This work was also supported in part by Hong Kong Research Grant Council grants 12300218, 12300519, 17201020, 17300021, C1013-21GF, C7004-21GF and Joint grant NSFC-RGC N-HKU76921. Finally, this work was supported in part by HKRGC grants CUHK14301718, CityU11301120, C1013-21GF, CityU 11303821, and CityU 9380101.
Keywords
- deep CNN
- plug-and-play
- spherical image inpainting