Learning Gated Non-local Residual for Single-image Rain Streak Removal

Lei ZHU, Zijun DENG, Xiaowei HU, Haoran XIE, Xuemiao XU, Jing QIN, Pheng-ann HENG

Research output: Journal PublicationsJournal Article (refereed)peer-review

36 Citations (Scopus)


This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.
Original languageEnglish
Article number9187841
Pages (from-to)2147-2159
Number of pages13
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number6
Early online date8 Sept 2020
Publication statusPublished - Jun 2021

Bibliographical note

This work was supported by a grant from National Natural Science Foundation of China (Grant No. 61902275), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No.: CUHK 14201620], a grant from the CUHK Direct Grant for Research 2018/19, a grant from The Hong Kong Polytechnic University (Project no. PolyU 152009/18E), a grant from the HKIBS Research Seed Fund 2019/20 (190-009), a grant from the Research Seed Fund (102367) of Lingnan University, Hong Kong, the grants from Key-Area Research and Development Program of Guangdong Province, China (2020B010165004, 2020B010166003, 2018B010107003), a grant from Guangdong High-level personnel program (2016TQ03X319), a grant from Guangzhou key project in industrial technology (201802010027), and the grants from NSFC (61772206, U1611461, 61472145).


  • gated learning
  • non-local residual learning
  • Single-image rain streak removal


Dive into the research topics of 'Learning Gated Non-local Residual for Single-image Rain Streak Removal'. Together they form a unique fingerprint.

Cite this