Learning Hierarchical Color Guidance for Depth Map Super-Resolution

Runmin CONG, Ronghui SHENG, Hao WU, Yulan GUO, Yunchao WEI, Wangmeng ZUO, Yao ZHAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

The color information are the most commonly used prior knowledge for depth map super-resolution (DSR), which can provide high-frequency boundary guidance for detail restoration. However, its role and functionality in DSR have not been fully developed. In this article, we rethink the utilization of color information and propose a hierarchical color guidance network (HCGNet) to achieve DSR. On the one hand, the low-level detail embedding (LDE) module is designed to supplement high-frequency color information of depth features in a residual mask manner at the low-level stages. On the other hand, the high-level abstract guidance (HAG) module is proposed to maintain semantic consistency in the reconstruction process by using a semantic mask that encodes the global guidance information. The color information of these 2-D plays a role in the front and back ends of the attention-based feature projection (AFP) module in a more comprehensive form. Simultaneously, the AFP module integrates the multiscale content enhancement (MCE) block and adaptive attention projection (AAP) block to make full use of multiscale information and adaptively project critical restoration information in an attention manner for DSR. Compared with the state-of-the-art methods on four benchmark datasets, our method achieves more competitive performance both qualitatively and quantitatively.

Original languageEnglish
Article number6503013
Number of pages13
JournalIEEE Transactions on Instrumentation and Measurement
Volume73
DOIs
Publication statusPublished - 25 Mar 2024

Bibliographical note

Publisher Copyright:
IEEE

Keywords

  • Adaptive projection
  • depth map
  • hierarchical color guidance
  • residual mask
  • semantic mask
  • super-resolution

Fingerprint

Dive into the research topics of 'Learning Hierarchical Color Guidance for Depth Map Super-Resolution'. Together they form a unique fingerprint.

Cite this