TY - JOUR
T1 - Learning Gated Non-local Residual for Single-image Rain Streak Removal
AU - ZHU, Lei
AU - DENG, Zijun
AU - HU, Xiaowei
AU - XIE, Haoran
AU - XU, Xuemiao
AU - QIN, Jing
AU - HENG, Pheng-ann
N1 - This work was supported by a grant from National Natural Science Foundation of China (Grant No. 61902275), a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No.: CUHK 14201620], a grant from the CUHK Direct Grant for Research 2018/19, a grant from The Hong Kong Polytechnic University (Project no. PolyU 152009/18E), a grant from the HKIBS Research Seed Fund 2019/20 (190-009), a grant from the Research Seed Fund (102367) of Lingnan University, Hong Kong, the grants from Key-Area Research and Development Program of Guangdong Province, China (2020B010165004, 2020B010166003, 2018B010107003), a grant from Guangdong High-level personnel program (2016TQ03X319), a grant from Guangzhou key project in industrial technology (201802010027), and the grants from NSFC (61772206, U1611461, 61472145).
PY - 2020/9/8
Y1 - 2020/9/8
N2 - This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.
AB - This work presents a gated non-local deep residual learning framework for image deraining. It can avoid the over-deraining or under-deraining caused by the global residual learning in existing deraining networks, since the learned soft gate in our method adaptively adjusts the amount of global residual to be passed for generating the final derained result. To generate feature maps for global residual prediction, we develop a non-local guided attention module (NLAM), which first obtains non-local features by exploiting spatial inter-dependencies among all the feature positions of local features produced by convolutional neural network (CNN), and then leverages the attention mechanism to merge the local and non-local features based on their complementary relation. Moreover, we develop a channel-wise gated prediction module to learn a soft gate on the global residual by explicitly modelling channel inter-dependencies of the feature maps obtained from NLAM. Experiments on four deraining benchmark datasets and real-world rainy images show that our network has a quantitative and qualitative improvement over state-of-the-arts.
U2 - 10.1109/TCSVT.2020.3022707
DO - 10.1109/TCSVT.2020.3022707
M3 - Journal Article (refereed)
JO - IEEE Transactions on Circuits and Systems for Video Technology
JF - IEEE Transactions on Circuits and Systems for Video Technology
SN - 1051-8215
ER -