Light Field Spatial Super-Resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization

Jing JIN, Junhui HOU*, Jie CHEN, Sam KWONG

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

104 Citations (Scopus)

Abstract

Light field (LF) images acquired by hand-held devices usually suffer from low spatial resolution as the limited sampling resources have to be shared with the angular dimension. LF spatial super-resolution (SR) thus becomes an indispensable part of the LF camera processing pipeline. The high-dimensionality characteristic and complex geometrical structure of LF images makes the problem more challenging than traditional single-image SR. The performance of existing methods are still limited as they fail to thoroughly explore the coherence among LF views and are insufficient in accurately preserving the parallax structure of the scene. In this paper, we propose a novel learning-based LF spatial SR framework, in which each view of an LF image is first individually super-resolved by exploring the complementary information among views with combinatorial geometry embedding. For accurate preservation of the parallax structure among the reconstructed views, a regularization network trained over a structure-aware loss function is subsequently appended to enforce correct parallax relationships over the intermediate estimation. Our proposed approach is evaluated over datasets with a large number of testing images including both synthetic and real-world scenes. Experimental results demonstrate the advantage of our approach over state-of-the-art methods, i.e., our method not only improves the average PSNR by more than 1.0 dB but also preserves more accurate parallax details, at a lower computation cost.

Original languageEnglish
Title of host publicationProceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020
PublisherIEEE
Pages2257-2266
Number of pages10
ISBN (Electronic)9781728171685
ISBN (Print)9781728171692
DOIs
Publication statusPublished - 2020
Externally publishedYes
Event2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition - Virtual, Online, United States
Duration: 14 Jun 202019 Jun 2020

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919

Conference

Conference2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2020
Country/TerritoryUnited States
CityVirtual, Online
Period14/06/2019/06/20

Bibliographical note

Funding Information:
This work was supported in part by the Hong Kong Research Grants Council under grants 9048123 and 9042820, and in part by the Basic Research General Program of Shenzhen Municipality under grants JCYJ20190808183003968.

Publisher Copyright:
© 2020 IEEE.

Fingerprint

Dive into the research topics of 'Light Field Spatial Super-Resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization'. Together they form a unique fingerprint.

Cite this