Relationship between generalization and diversity in coevolutionary learning

Siang Yew CHONG, Peter TIŇO, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

33 Citations (Scopus)

Abstract

Games have long played an important role in the development and understanding of coevolutionary learning systems. In particular, the search process in coevolutionary learning is guided by strategic interactions between solutions in the population, which can be naturally framed as game playing. We study two important issues in coevolutionary learninggeneralization performance and diversityusing games. The first one is concerned with the coevolutionary learning of strategies with high generalization performance, that is, strategies that can outperform against a large number of test strategies (opponents) that may not have been seen during coevolution. The second one is concerned with diversity levels in the population that may lead to the search of strategies with poor generalization performance. It is not known if there is a relationship between generalization and diversity in coevolutionary learning. This paper investigates whether there is such a relationship in coevolutionary learning through a detailed empirical study. We systematically investigate the impact of various diversity maintenance approaches on the generalization performance of coevolutionary learning quantitatively using case studies. The problem of the iterated prisoner's dilemma (IPD) game is considered. Unlike past studies, we can measure both the generalization performance and the diversity level of the population of evolved strategies. Results from our case studies show that the introduction and maintenance of diversity do not necessarily lead to the coevolutionary learning of strategies with high generalization performance. However, if individual strategies can be combined (e.g., using a gating mechanism), there is the potential of exploiting diversity in coevolutionary learning to improve generalization performance. Specifically, when the introduction and maintenance of diversity lead to a speciated population during coevolution, where each specialist strategy is capable of outperforming different opponents, the population as a whole can have a significantly higher generalization performance compared to individual strategies. © 2009 IEEE.
Original languageEnglish
Article number5286208
Pages (from-to)214-232
Number of pages19
JournalIEEE Transactions on Games
Volume1
Issue number3
DOIs
Publication statusPublished - Sept 2009
Externally publishedYes

Bibliographical note

The work of X. Yao was supported in part by EPSRC under Grant GR/T10671/01.

Keywords

  • Coevolutionary learning
  • Diversity maintenance
  • Evolutionary computation (EC)
  • Generalization performance
  • Iterated Prisoner's dilemma (IPD)

Fingerprint

Dive into the research topics of 'Relationship between generalization and diversity in coevolutionary learning'. Together they form a unique fingerprint.

Cite this