AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

Marcos V. CONDE; Saman ZADTOOTAGHAJ; Nabajeet BARMAN; Radu TIMOFTE; Chenlong HE; Qi ZHENG; Ruoxi ZHU; Zhengzhong TU; Haiqiang WANG; Xiangguang CHEN; Wenhui MENG; Xiang PAN; Huiying SHI; Han ZHU; Xiaozhong XU; Lei SUN; Zhenzhong CHEN; Shan LIU; Zicheng ZHANG; Haoning WU; Yingjie ZHOU; Chunyi LI; Xiaohong LIU; Weisi LIN; Guangtao ZHAI; Wei SUN; Yuqin CAO; Yanwei JIANG; Jun JIA; Zhichao ZHANG; Zijian CHEN; Weixia ZHANG; Xiongkuo MIN; Steve GORING; Zihao QI; Chen FENG

doi:10.1109/CVPRW63382.2024.00591

AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

Marcos V. CONDE^*
, Saman ZADTOOTAGHAJ
, Nabajeet BARMAN
, Radu TIMOFTE
, Chenlong HE
, Qi ZHENG
, Ruoxi ZHU
, Zhengzhong TU
, Haiqiang WANG
, Xiangguang CHEN
, Wenhui MENG
, Xiang PAN
, Huiying SHI
, Han ZHU
, Xiaozhong XU
, Lei SUN
, Zhenzhong CHEN
, Shan LIU
, Zicheng ZHANG
, Haoning WU

Yingjie ZHOU, Chunyi LI, Xiaohong LIU, Weisi LIN, Guangtao ZHAI, Wei SUN, Yuqin CAO, Yanwei JIANG, Jun JIA, Zhichao ZHANG, Zijian CHEN, Weixia ZHANG, Xiongkuo MIN, Steve GORING, Zihao QI, Chen FENG

^*Corresponding author for this work

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

9 Citations (Scopus)

Abstract

This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted results during the challenge period. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for Video Quality Assessment of user-generated content.

Original language	English
Title of host publication	Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024
Publisher	IEEE
Pages	5826-5837
Number of pages	12
ISBN (Electronic)	9798350365474
ISBN (Print)	9798350365481
DOIs	https://doi.org/10.1109/CVPRW63382.2024.00591
Publication status	Published - 17 Jun 2024
Externally published	Yes
Event	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024 - Seattle, United States Duration: 16 Jun 2024 → 22 Jun 2024

Publication series

Name	IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (Print)	2160-7508
ISSN (Electronic)	2160-7516

Conference

Conference	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024
Country/Territory	United States
City	Seattle
Period	16/06/24 → 22/06/24

Bibliographical note

Publisher Copyright:
© 2024 IEEE.

Funding

This work was partially supported by the Humboldt Foundation. We thank the AIS 2024 sponsors: Meta Reality Labs, Meta, Netflix, Sony Interactive Entertainment (FTG), and the University of Würzburg (Computer Vision Lab). The challenge organizers thank Ioannis Katsavounidis (Meta), Christos Bampis (Netflix), and Balu Adsumilli (Google) for their feedback.

Keywords

AIS
Image Quality Assessment
IQA
video
video quality assessment
VQA

Access to Document

10.1109/CVPRW63382.2024.00591

Cite this

CONDE, M. V., ZADTOOTAGHAJ, S., BARMAN, N., TIMOFTE, R., HE, C., ZHENG, Q., ZHU, R., TU, Z., WANG, H., CHEN, X., MENG, W., PAN, X., SHI, H., ZHU, H., XU, X., SUN, L., CHEN, Z., LIU, S., ZHANG, Z., ... FENG, C. (2024). AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results. In Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024 (pp. 5826-5837). (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). IEEE. https://doi.org/10.1109/CVPRW63382.2024.00591

CONDE, Marcos V. ; ZADTOOTAGHAJ, Saman ; BARMAN, Nabajeet et al. / AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results. Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024. IEEE, 2024. pp. 5826-5837 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).

@inproceedings{cda7ca877a7647fea1f16441035688de,

title = "AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results",

abstract = "This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted results during the challenge period. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for Video Quality Assessment of user-generated content.",

keywords = "AIS, Image Quality Assessment, IQA, video, video quality assessment, VQA",

author = "CONDE, \{Marcos V.\} and Saman ZADTOOTAGHAJ and Nabajeet BARMAN and Radu TIMOFTE and Chenlong HE and Qi ZHENG and Ruoxi ZHU and Zhengzhong TU and Haiqiang WANG and Xiangguang CHEN and Wenhui MENG and Xiang PAN and Huiying SHI and Han ZHU and Xiaozhong XU and Lei SUN and Zhenzhong CHEN and Shan LIU and Zicheng ZHANG and Haoning WU and Yingjie ZHOU and Chunyi LI and Xiaohong LIU and Weisi LIN and Guangtao ZHAI and Wei SUN and Yuqin CAO and Yanwei JIANG and Jun JIA and Zhichao ZHANG and Zijian CHEN and Weixia ZHANG and Xiongkuo MIN and Steve GORING and Zihao QI and Chen FENG",

note = "Publisher Copyright: {\textcopyright} 2024 IEEE.; 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024 ; Conference date: 16-06-2024 Through 22-06-2024",

year = "2024",

month = jun,

day = "17",

doi = "10.1109/CVPRW63382.2024.00591",

language = "English",

isbn = "9798350365481",

series = "IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops",

publisher = "IEEE",

pages = "5826--5837",

booktitle = "Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024",

}

CONDE, MV, ZADTOOTAGHAJ, S, BARMAN, N, TIMOFTE, R, HE, C, ZHENG, Q, ZHU, R, TU, Z, WANG, H, CHEN, X, MENG, W, PAN, X, SHI, H, ZHU, H, XU, X, SUN, L, CHEN, Z, LIU, S, ZHANG, Z, WU, H, ZHOU, Y, LI, C, LIU, X, LIN, W, ZHAI, G, SUN, W, CAO, Y, JIANG, Y, JIA, J, ZHANG, Z, CHEN, Z, ZHANG, W, MIN, X, GORING, S, QI, Z & FENG, C 2024, AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results. in Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, IEEE, pp. 5826-5837, 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024, Seattle, United States, 16/06/24. https://doi.org/10.1109/CVPRW63382.2024.00591

AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results. / CONDE, Marcos V.; ZADTOOTAGHAJ, Saman; BARMAN, Nabajeet et al.
Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024. IEEE, 2024. p. 5826-5837 (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops).

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

TY - GEN

T1 - AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

AU - CONDE, Marcos V.

AU - ZADTOOTAGHAJ, Saman

AU - BARMAN, Nabajeet

AU - TIMOFTE, Radu

AU - HE, Chenlong

AU - ZHENG, Qi

AU - ZHU, Ruoxi

AU - TU, Zhengzhong

AU - WANG, Haiqiang

AU - CHEN, Xiangguang

AU - MENG, Wenhui

AU - PAN, Xiang

AU - SHI, Huiying

AU - ZHU, Han

AU - XU, Xiaozhong

AU - SUN, Lei

AU - CHEN, Zhenzhong

AU - LIU, Shan

AU - ZHANG, Zicheng

AU - WU, Haoning

AU - ZHOU, Yingjie

AU - LI, Chunyi

AU - LIU, Xiaohong

AU - LIN, Weisi

AU - ZHAI, Guangtao

AU - SUN, Wei

AU - CAO, Yuqin

AU - JIANG, Yanwei

AU - JIA, Jun

AU - ZHANG, Zhichao

AU - CHEN, Zijian

AU - ZHANG, Weixia

AU - MIN, Xiongkuo

AU - GORING, Steve

AU - QI, Zihao

AU - FENG, Chen

PY - 2024/6/17

Y1 - 2024/6/17

N2 - This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted results during the challenge period. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for Video Quality Assessment of user-generated content.

AB - This paper reviews the AIS 2024 Video Quality Assessment (VQA) Challenge, focused on User-Generated Content (UGC). The aim of this challenge is to gather deep learning-based methods capable of estimating the perceptual quality of UGC videos. The user-generated videos from the YouTube UGC Dataset include diverse content (sports, games, lyrics, anime, etc.), quality and resolutions. The proposed methods must process 30 FHD frames under 1 second. In the challenge, a total of 102 participants registered, and 15 submitted results during the challenge period. The performance of the top-5 submissions is reviewed and provided here as a survey of diverse deep models for Video Quality Assessment of user-generated content.

KW - AIS

KW - Image Quality Assessment

KW - IQA

KW - video

KW - video quality assessment

KW - VQA

UR - https://www.scopus.com/pages/publications/85193082727

U2 - 10.1109/CVPRW63382.2024.00591

DO - 10.1109/CVPRW63382.2024.00591

M3 - Conference paper (refereed)

AN - SCOPUS:85193082727

SN - 9798350365481

T3 - IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

SP - 5826

EP - 5837

BT - Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024

PB - IEEE

T2 - 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024

Y2 - 16 June 2024 through 22 June 2024

ER -

CONDE MV, ZADTOOTAGHAJ S, BARMAN N, TIMOFTE R, HE C, ZHENG Q et al. AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results. In Proceedings: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2024. IEEE. 2024. p. 5826-5837. (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). doi: 10.1109/CVPRW63382.2024.00591

AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this