Real-world rain is a mixture of rain streaks and rainy haze. However, current efforts formulate image rain streaks removal and rainy haze removal as separated models, worsening the loss of image details. This paper attempts to solve the mixture of rain removal problem in a single model by estimating the scene depths of images. To this end, we propose a novel SEMI-supervised Mixture Of rain REmoval Generative Adversarial Network (Semi-MoreGAN). Unlike most of existing methods, Semi-MoreGAN is a joint learning paradigm of mixture of rain removal and depth estimation; and it effectively integrates the image features with the depth information for better rain removal. Furthermore, it leverages unpaired real-world rainy and clean images to bridge the gap between synthetic and real-world rain. Extensive experiments show clear improvements of our approach over twenty representative state-of-the-arts on both synthetic and real-world rainy images. Source code is available at https://github.com/syy-whu/Semi-MoreGAN.
Bibliographical noteFunding Information:
This work was supported by the National Key Research and Development Program of China (No. 2021ZD0113200, No. 2020YFB1805400), the National Natural Science Foundation of China (No. 42071431, No. 62172218), the Provincial Key Research and Development Program of Hubei, China (No. 2020BAB101), the Direct Grant (No. DR22A2) and the Research Grant entitled “Self‐Supervised Learning for Medical Images” (No. 871228) of Lingnan University, Hong Kong.
© 2022 The Author(s) Computer Graphics Forum © 2022 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd.
- CCS Concepts
- • Computing methodologies → Image Processing