This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.
|Number of pages||12|
|Journal||IEEE Transactions on Circuits and Systems for Video Technology|
|Publication status||E-pub ahead of print - 8 Sep 2020|
Bibliographical noteThis work was supported by a grant from National Natural Science Foundation of China (Grant No. 61902275), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No.: CUHK 14201620], a grant from the CUHK Direct Grant for Research 2018/19, a grant from The Hong Kong Polytechnic University (Project no. PolyU 152009/18E), a grant from the HKIBS Research Seed Fund 2019/20 (190-009), a grant from the Research Seed Fund (102367) of Lingnan University, Hong Kong, the grants from Key-Area Research and Development Program of Guangdong Province, China (2020B010165004, 2020B010166003, 2018B010107003), a grant from Guangdong High-level personnel program (2016TQ03X319), a grant from Guangzhou key project in industrial technology (201802010027), and the grants from NSFC (61772206, U1611461, 61472145).
ZHU, L., DENG, Z., HU, X., XIE, H., XU, X., QIN, J., & HENG, P. (2020). Learning Gated Non-local Residual for Single-image Rain Streak Removal. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2020.3022707