BridgeNet : A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation

Qi TANG, Runmin CONG, Ronghui SHENG, Lingzhi HE, Dan ZHANG, Yao ZHAO, Sam KWONG

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

28 Citations (Scopus)

Abstract

Depth map super-resolution is a task with high practical application requirements in the industry. Existing color-guided depth map super-resolution methods usually necessitate an extra branch to extract high-frequency detail information from RGB image to guide the low-resolution depth map reconstruction. However, because there are still some differences between the two modalities, direct information transmission in the feature dimension or edge map dimension cannot achieve satisfactory result, and may even trigger texture copying in areas where the structures of the RGB-D pair are inconsistent. Inspired by the multi-task learning, we propose a joint learning network of depth map super-resolution (DSR) and monocular depth estimation (MDE) without introducing additional supervision labels. For the interaction of two subnetworks, we adopt a differentiated guidance strategy and design two bridges correspondingly. One is the high-frequency attention bridge (HABdg) designed for the feature encoding process, which learns the high-frequency information of the MDE task to guide the DSR task. The other is the content guidance bridge (CGBdg) designed for the depth map reconstruction process, which provides the content guidance learned from DSR task for MDE task. The entire network architecture is highly portable and can provide a paradigm for associating the DSR and MDE tasks. Extensive experiments on benchmark datasets demonstrate that our method achieves competitive performance. Our code and models are available at https://rmcong.github.io/proj_BridgeNet.html.
Original languageEnglish
Title of host publicationProceedings of the 29th ACM International Conference on Multimedia
Place of PublicationUnited States
PublisherAssociation for Computing Machinery
Pages2148-2157
ISBN (Print)9781450386517
DOIs
Publication statusPublished - 2021
Externally publishedYes
EventThe 29th ACM International Conference on Multimedia - Virtual, Online, China
Duration: 20 Oct 202124 Oct 2021

Conference

ConferenceThe 29th ACM International Conference on Multimedia
Country/TerritoryChina
CityVirtual, Online
Period20/10/2124/10/21

Bibliographical note

This work was supported by the Beijing Nova Program under Grant Z201100006820016, in part by the National Key Research and Development of China under Grant 2018AAA0102100, in part by the National Natural Science Foundation of China under Grant 62002014, Grant U1936212, in part by Elite Scientist Sponsorship Program by the China Association for Science and Technology under Grant 2020QNRC001, in part by General Research Fund-Research Grants Council (GRF-RGC) under Grant 9042816 (CityU 11209819), Grant 9042958 (CityU 11203820), in part by Hong Kong Scholars Program under Grant XJ2020040, in part by CAAI-Huawei MindSpore Open Fund, and in part by China Postdoctoral Science Foundation under Grant 2020T130050, Grant 2019M660438.

Keywords

  • depth map
  • monocular depth estimation
  • multi-task learning
  • super-resolution

Fingerprint

Dive into the research topics of 'BridgeNet : A Joint Learning Network of Depth Map Super-Resolution and Monocular Depth Estimation'. Together they form a unique fingerprint.

Cite this