The comparative impact of critics and consumers : applying the generalisability theory to online movie ratings

Ling PENG, Geng CUI, Chunyu LI

Research output: Journal PublicationsJournal Article (refereed)Researchpeer-review

5 Citations (Scopus)

Abstract

This study employs a new measurement theory (i.e. Generalisability Theory) to investigate the comparative influence of early movie ratings from professional critics versus ordinary consumers on latent movie performance. The empirical results show that both ordinary consumers and critics have great impact on the latent movie performance. In particular, the main effect of rater sources and the two-way interaction between raters and movies are substantial contributors to the variation in movie performance, with the contribution from ordinary consumers even more substantial than that from professional critics. However, professional critics provide more reliable ratings (a higher G coefficient) than ordinary consumers. Moreover, we found that genre familiarity is an important factor that moderates the differential effect of these two sources of ratings. Professional critic ratings contribute more to the total variance of movie performance evaluations in the case of less familiar genres, while ordinary consumer ratings contribute more to that in the case of more familiar genres. The aggregate level validity (correlation) results for each rater source indicate that professional critics consistently provide better concurrent and predictive validity than ordinary consumers. While our analyses focused on the impact of two sources of ratings on movie performance evaluations, the findings have implications not limited to the movie industry. They are also applicable to the broad category of experience goods such as music, restaurants, video games and books, where consumers could seek opinion from both experts and ordinary consumers.
Original languageEnglish
Pages (from-to)413-436
Number of pages24
JournalInternational Journal of Market Research
Volume55
Issue number3
Early online date1 May 2013
DOIs
Publication statusPublished - 1 May 2013

Fingerprint

Rating
Movies
Performance evaluation
Music
Restaurants
Factors
Coefficients
Familiarity
Video games
Interaction
Empirical results
Predictive validity
Film industry

Cite this

@article{58e8bcedac5b42409dd63f397112b9b7,
title = "The comparative impact of critics and consumers : applying the generalisability theory to online movie ratings",
abstract = "This study employs a new measurement theory (i.e. Generalisability Theory) to investigate the comparative influence of early movie ratings from professional critics versus ordinary consumers on latent movie performance. The empirical results show that both ordinary consumers and critics have great impact on the latent movie performance. In particular, the main effect of rater sources and the two-way interaction between raters and movies are substantial contributors to the variation in movie performance, with the contribution from ordinary consumers even more substantial than that from professional critics. However, professional critics provide more reliable ratings (a higher G coefficient) than ordinary consumers. Moreover, we found that genre familiarity is an important factor that moderates the differential effect of these two sources of ratings. Professional critic ratings contribute more to the total variance of movie performance evaluations in the case of less familiar genres, while ordinary consumer ratings contribute more to that in the case of more familiar genres. The aggregate level validity (correlation) results for each rater source indicate that professional critics consistently provide better concurrent and predictive validity than ordinary consumers. While our analyses focused on the impact of two sources of ratings on movie performance evaluations, the findings have implications not limited to the movie industry. They are also applicable to the broad category of experience goods such as music, restaurants, video games and books, where consumers could seek opinion from both experts and ordinary consumers.",
author = "Ling PENG and Geng CUI and Chunyu LI",
year = "2013",
month = "5",
day = "1",
doi = "10.2501/IJMR-2013-037",
language = "English",
volume = "55",
pages = "413--436",
journal = "International Journal of Market Research",
issn = "1470-7853",
publisher = "Market Research Society",
number = "3",

}

The comparative impact of critics and consumers : applying the generalisability theory to online movie ratings. / PENG, Ling; CUI, Geng; LI, Chunyu.

In: International Journal of Market Research, Vol. 55, No. 3, 01.05.2013, p. 413-436.

Research output: Journal PublicationsJournal Article (refereed)Researchpeer-review

TY - JOUR

T1 - The comparative impact of critics and consumers : applying the generalisability theory to online movie ratings

AU - PENG, Ling

AU - CUI, Geng

AU - LI, Chunyu

PY - 2013/5/1

Y1 - 2013/5/1

N2 - This study employs a new measurement theory (i.e. Generalisability Theory) to investigate the comparative influence of early movie ratings from professional critics versus ordinary consumers on latent movie performance. The empirical results show that both ordinary consumers and critics have great impact on the latent movie performance. In particular, the main effect of rater sources and the two-way interaction between raters and movies are substantial contributors to the variation in movie performance, with the contribution from ordinary consumers even more substantial than that from professional critics. However, professional critics provide more reliable ratings (a higher G coefficient) than ordinary consumers. Moreover, we found that genre familiarity is an important factor that moderates the differential effect of these two sources of ratings. Professional critic ratings contribute more to the total variance of movie performance evaluations in the case of less familiar genres, while ordinary consumer ratings contribute more to that in the case of more familiar genres. The aggregate level validity (correlation) results for each rater source indicate that professional critics consistently provide better concurrent and predictive validity than ordinary consumers. While our analyses focused on the impact of two sources of ratings on movie performance evaluations, the findings have implications not limited to the movie industry. They are also applicable to the broad category of experience goods such as music, restaurants, video games and books, where consumers could seek opinion from both experts and ordinary consumers.

AB - This study employs a new measurement theory (i.e. Generalisability Theory) to investigate the comparative influence of early movie ratings from professional critics versus ordinary consumers on latent movie performance. The empirical results show that both ordinary consumers and critics have great impact on the latent movie performance. In particular, the main effect of rater sources and the two-way interaction between raters and movies are substantial contributors to the variation in movie performance, with the contribution from ordinary consumers even more substantial than that from professional critics. However, professional critics provide more reliable ratings (a higher G coefficient) than ordinary consumers. Moreover, we found that genre familiarity is an important factor that moderates the differential effect of these two sources of ratings. Professional critic ratings contribute more to the total variance of movie performance evaluations in the case of less familiar genres, while ordinary consumer ratings contribute more to that in the case of more familiar genres. The aggregate level validity (correlation) results for each rater source indicate that professional critics consistently provide better concurrent and predictive validity than ordinary consumers. While our analyses focused on the impact of two sources of ratings on movie performance evaluations, the findings have implications not limited to the movie industry. They are also applicable to the broad category of experience goods such as music, restaurants, video games and books, where consumers could seek opinion from both experts and ordinary consumers.

UR - http://commons.ln.edu.hk/sw_master/2245

UR - https://www.scopus.com/inward/record.uri?eid=2-s2.0-84877983106&doi=10.2501%2fIJMR-2013-037&partnerID=40&md5=d1a524b60a7936f9776a00e3a67b6854

U2 - 10.2501/IJMR-2013-037

DO - 10.2501/IJMR-2013-037

M3 - Journal Article (refereed)

VL - 55

SP - 413

EP - 436

JO - International Journal of Market Research

JF - International Journal of Market Research

SN - 1470-7853

IS - 3

ER -