View-spatial-temporal post-refinement for view synthesis in 3D video systems

Linwei ZHU; Yun ZHANG; Mei YU; Gangyi JIANG; Sam KWONG

doi:10.1016/j.image.2013.08.005

View-spatial-temporal post-refinement for view synthesis in 3D video systems

Linwei ZHU, Yun ZHANG, Mei YU, Gangyi JIANG, Sam KWONG

Research output: Journal Publications › Journal Article (refereed) › peer-review

10 Citations (Scopus)

Abstract

Depth image based rendering is one of key techniques to realize view synthesis for three-dimensional television and free-viewpoint television, which provide high quality and immersive experiences to end viewers. However, artifacts of rendered images, including holes caused by occlusion/disclosure and boundary artifacts, may degrade the subjective and objective image quality. To handle these problems and improve the quality of rendered images, we present a novel view-spatial-temporal post-refinement method for view synthesis, in which new hole-filling and boundary artifact removal techniques are proposed. In addition, we propose an optimal reference frame selection algorithm for a better trade-off between the computational complexity and rendered image quality. Experimental results show that the proposed method can achieve a peak signal-to-noise ratio gain of 0.94 dB on average for multiview video test sequences when compared with the benchmark view synthesis reference software. In addition, the subjective quality of the rendered image is also improved. © 2013 Elsevier B.V.

Original language	English
Pages (from-to)	1342-1357
Journal	Signal Processing: Image Communication
Volume	28
Issue number	10
Early online date	29 Aug 2013
DOIs	https://doi.org/10.1016/j.image.2013.08.005
Publication status	Published - Nov 2013
Externally published	Yes

Funding

Thanks to Microsoft Research, HHI, Nagoya University and Poznan University of technology for providing multiview video sequence with fine quality depth information and camera parameters. This work was supported by National Natural Science Foundation of China under Grants 61102088, 61111140392 and 61271270, and in party by Guangdong Provincial Natural Science Foundation under Grant S2012010008457, Shenzhen Emerging Industries of the Strategic Basic Research Grant under Grant JCYJ20120617151719115, and the Innovation Fund Project for Graduate Student of Zhejiang Province ( YK2010047 ).

Keywords

Boundary artifact
Depth image based rendering
Hole-filling
Multiview video
View synthesis

Access to Document

10.1016/j.image.2013.08.005

Cite this

@article{17176bcf096449498cb176d9bda4e92a,

title = "View-spatial-temporal post-refinement for view synthesis in 3D video systems",

abstract = "Depth image based rendering is one of key techniques to realize view synthesis for three-dimensional television and free-viewpoint television, which provide high quality and immersive experiences to end viewers. However, artifacts of rendered images, including holes caused by occlusion/disclosure and boundary artifacts, may degrade the subjective and objective image quality. To handle these problems and improve the quality of rendered images, we present a novel view-spatial-temporal post-refinement method for view synthesis, in which new hole-filling and boundary artifact removal techniques are proposed. In addition, we propose an optimal reference frame selection algorithm for a better trade-off between the computational complexity and rendered image quality. Experimental results show that the proposed method can achieve a peak signal-to-noise ratio gain of 0.94 dB on average for multiview video test sequences when compared with the benchmark view synthesis reference software. In addition, the subjective quality of the rendered image is also improved. {\textcopyright} 2013 Elsevier B.V.",

keywords = "Boundary artifact, Depth image based rendering, Hole-filling, Multiview video, View synthesis",

author = "Linwei ZHU and Yun ZHANG and Mei YU and Gangyi JIANG and Sam KWONG",

year = "2013",

month = nov,

doi = "10.1016/j.image.2013.08.005",

language = "English",

volume = "28",

pages = "1342--1357",

journal = "Signal Processing: Image Communication",

issn = "0923-5965",

publisher = "Elsevier B.V.",

number = "10",

}

TY - JOUR

T1 - View-spatial-temporal post-refinement for view synthesis in 3D video systems

AU - ZHU, Linwei

AU - ZHANG, Yun

AU - YU, Mei

AU - JIANG, Gangyi

AU - KWONG, Sam

PY - 2013/11

Y1 - 2013/11

N2 - Depth image based rendering is one of key techniques to realize view synthesis for three-dimensional television and free-viewpoint television, which provide high quality and immersive experiences to end viewers. However, artifacts of rendered images, including holes caused by occlusion/disclosure and boundary artifacts, may degrade the subjective and objective image quality. To handle these problems and improve the quality of rendered images, we present a novel view-spatial-temporal post-refinement method for view synthesis, in which new hole-filling and boundary artifact removal techniques are proposed. In addition, we propose an optimal reference frame selection algorithm for a better trade-off between the computational complexity and rendered image quality. Experimental results show that the proposed method can achieve a peak signal-to-noise ratio gain of 0.94 dB on average for multiview video test sequences when compared with the benchmark view synthesis reference software. In addition, the subjective quality of the rendered image is also improved. © 2013 Elsevier B.V.

AB - Depth image based rendering is one of key techniques to realize view synthesis for three-dimensional television and free-viewpoint television, which provide high quality and immersive experiences to end viewers. However, artifacts of rendered images, including holes caused by occlusion/disclosure and boundary artifacts, may degrade the subjective and objective image quality. To handle these problems and improve the quality of rendered images, we present a novel view-spatial-temporal post-refinement method for view synthesis, in which new hole-filling and boundary artifact removal techniques are proposed. In addition, we propose an optimal reference frame selection algorithm for a better trade-off between the computational complexity and rendered image quality. Experimental results show that the proposed method can achieve a peak signal-to-noise ratio gain of 0.94 dB on average for multiview video test sequences when compared with the benchmark view synthesis reference software. In addition, the subjective quality of the rendered image is also improved. © 2013 Elsevier B.V.

KW - Boundary artifact

KW - Depth image based rendering

KW - Hole-filling

KW - Multiview video

KW - View synthesis

UR - http://www.scopus.com/inward/record.url?scp=84888205812&partnerID=8YFLogxK

U2 - 10.1016/j.image.2013.08.005

DO - 10.1016/j.image.2013.08.005

M3 - Journal Article (refereed)

SN - 0923-5965

VL - 28

SP - 1342

EP - 1357

JO - Signal Processing: Image Communication

JF - Signal Processing: Image Communication

IS - 10

ER -

View-spatial-temporal post-refinement for view synthesis in 3D video systems

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this