SRNSD: Structure-Regularized Night-Time Self-Supervised Monocular Depth Estimation for Outdoor Scenes

Runmin CONG, Chunlei WU, Xibin SONG*, Wei ZHANG, Sam KWONG, Hongdong LI, Pan JI

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

Deep CNNs have achieved impressive improvements for night-time self-supervised depth estimation form a monocular image. However, the performance degrades considerably compared to day-time depth estimation due to significant domain gaps, low visibility, and varying illuminations between day and night images. To address these challenges, we propose a novel night-time self-supervised monocular depth estimation framework with structure regularization, i.e., SRNSD, which incorporates three aspects of constraints for better performance, including feature and depth domain adaptation, image perspective constraint, and cropped multi-scale consistency loss. Specifically, we utilize adaptations of both feature and depth output spaces for better night-time feature extraction and depth map prediction, along with high- and low-frequency decoupling operations for better depth structure and texture recovery. Meanwhile, we employ an image perspective constraint to enhance the smoothness and obtain better depth maps in areas where the luminosity jumps change. Furthermore, we introduce a simple yet effective cropped multi-scale consistency loss that utilizes consistency among different scales of depth outputs for further optimization, refining the detailed textures and structures of predicted depth. Experimental results on different benchmarks with depth ranges of 40m and 60m, including Oxford RobotCar dataset, nuScenes dataset and CARLA-EPE dataset, demonstrate the superiority of our approach over state-of-the-art night-time self-supervised depth estimation approaches across multiple metrics, proving our effectiveness.

Original languageEnglish
Pages (from-to)5538-5550
Number of pages13
JournalIEEE Transactions on Image Processing
Volume33
Early online date26 Sept 2024
DOIs
Publication statusPublished - 2024

Bibliographical note

Publisher Copyright:
© 1992-2012 IEEE.

Funding

This work was supported in part by Tencent XR Vision Labs, in part by the National Natural Science Foundation of China under Grant 61991411 and Grant 62471278, in part by the National Science and Technology Major Project of China under Grant 2021ZD0112002, in part by Taishan Scholar Project of Shandong Province under Grant tsqn202306079, in part by Hong Kong GRF-RGC General Research Fund under Grant 11203820, in part by Xiaomi Young Talents Program, and in part by the Project for Self-Developed Innovation Team of Jinan City under Grant 2021GXRC038.

Keywords

  • Domain adaption
  • night-time depth estimation
  • structure regularization

Fingerprint

Dive into the research topics of 'SRNSD: Structure-Regularized Night-Time Self-Supervised Monocular Depth Estimation for Outdoor Scenes'. Together they form a unique fingerprint.

Cite this