Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing JIN; Junhui HOU; Jie CHEN; Henry YEUNG; Sam KWONG

doi:10.1109/ICDSP.2018.8631859

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing JIN, Junhui HOU, Jie CHEN, Henry YEUNG, Sam KWONG

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

4 Citations (Scopus)

Abstract

Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

Original language	English
Title of host publication	2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)
Publisher	IEEE
Number of pages	5
ISBN (Electronic)	9781538668115
ISBN (Print)	9781538668122
DOIs	https://doi.org/10.1109/ICDSP.2018.8631859
Publication status	Published - Nov 2018
Externally published	Yes
Event	23rd IEEE International Conference on Digital Signal Processing (DSP 2018) - ShanghaiTech University, Shanghai, China Duration: 19 Nov 2018 → 21 Nov 2018

Conference

Conference	23rd IEEE International Conference on Digital Signal Processing (DSP 2018)
Country/Territory	China
City	Shanghai
Period	19/11/18 → 21/11/18

Bibliographical note

This work was supported in part by the CityU Start-up Grant for New Faculty under Grant 7200537/CS and in part by the Hong Kong RGC Early Career Scheme Funds 9048123 (CityU 21211518).

Keywords

deep learning
Light field
super-resolution

Access to Document

10.1109/ICDSP.2018.8631859

Cite this

@inproceedings{19209e73334947a2a8e4c755205d3de2,

title = "Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image",

abstract = "Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.",

keywords = "deep learning, Light field, super-resolution",

author = "Jing JIN and Junhui HOU and Jie CHEN and Henry YEUNG and Sam KWONG",

note = "This work was supported in part by the CityU Start-up Grant for New Faculty under Grant 7200537/CS and in part by the Hong Kong RGC Early Career Scheme Funds 9048123 (CityU 21211518).; 23rd IEEE International Conference on Digital Signal Processing (DSP 2018)<br/> ; Conference date: 19-11-2018 Through 21-11-2018",

year = "2018",

month = nov,

doi = "10.1109/ICDSP.2018.8631859",

language = "English",

isbn = "9781538668122",

booktitle = "2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)",

publisher = "IEEE",

}

JIN, J, HOU, J, CHEN, J, YEUNG, H & KWONG, S 2018, Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. in 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)., 8631859, IEEE, 23rd IEEE International Conference on Digital Signal Processing (DSP 2018)
, Shanghai, China, 19/11/18. https://doi.org/10.1109/ICDSP.2018.8631859

TY - GEN

T1 - Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

AU - JIN, Jing

AU - HOU, Junhui

AU - CHEN, Jie

AU - YEUNG, Henry

AU - KWONG, Sam

N1 - This work was supported in part by the CityU Start-up Grant for New Faculty under Grant 7200537/CS and in part by the Hong Kong RGC Early Career Scheme Funds 9048123 (CityU 21211518).

PY - 2018/11

Y1 - 2018/11

N2 - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

AB - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

KW - deep learning

KW - Light field

KW - super-resolution

UR - http://www.scopus.com/inward/record.url?scp=85062776177&partnerID=8YFLogxK

U2 - 10.1109/ICDSP.2018.8631859

DO - 10.1109/ICDSP.2018.8631859

M3 - Conference paper (refereed)

SN - 9781538668122

BT - 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)

PB - IEEE

T2 - 23rd IEEE International Conference on Digital Signal Processing (DSP 2018)<br/>

Y2 - 19 November 2018 through 21 November 2018

ER -

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this