Abstract
Deep CNNs have achieved impressive improvements for night-time self-supervised depth estimation form a monocular image. However, the performance degrades considerably compared to day-time depth estimation due to significant domain gaps, low visibility, and varying illuminations between day and night images. To address these challenges, we propose a novel night-time self-supervised monocular depth estimation framework with structure regularization, i.e., SRNSD, which incorporates three aspects of constraints for better performance, including feature and depth domain adaptation, image perspective constraint, and cropped multi-scale consistency loss. Specifically, we utilize adaptations of both feature and depth output spaces for better night-time feature extraction and depth map prediction, along with high- and low-frequency decoupling operations for better depth structure and texture recovery. Meanwhile, we employ an image perspective constraint to enhance the smoothness and obtain better depth maps in areas where the luminosity jumps change. Furthermore, we introduce a simple yet effective cropped multi-scale consistency loss that utilizes consistency among different scales of depth outputs for further optimization, refining the detailed textures and structures of predicted depth. Experimental results on different benchmarks with depth ranges of 40m and 60m, including Oxford RobotCar dataset, nuScenes dataset and CARLA-EPE dataset, demonstrate the superiority of our approach over state-of-the-art night-time self-supervised depth estimation approaches across multiple metrics, proving our effectiveness.
Original language | English |
---|---|
Pages (from-to) | 5538-5550 |
Number of pages | 13 |
Journal | IEEE Transactions on Image Processing |
Volume | 33 |
Early online date | 26 Sept 2024 |
DOIs | |
Publication status | Published - 2024 |
Bibliographical note
Publisher Copyright:© 1992-2012 IEEE.
Funding
This work was supported in part by Tencent XR Vision Labs, in part by the National Natural Science Foundation of China under Grant 61991411 and Grant 62471278, in part by the National Science and Technology Major Project of China under Grant 2021ZD0112002, in part by Taishan Scholar Project of Shandong Province under Grant tsqn202306079, in part by Hong Kong GRF-RGC General Research Fund under Grant 11203820, in part by Xiaomi Young Talents Program, and in part by the Project for Self-Developed Innovation Team of Jinan City under Grant 2021GXRC038.
Keywords
- Domain adaption
- night-time depth estimation
- structure regularization