Evolving benchmark functions using kruskal-wallis test

Yang LOU, Shiu Yin YUEN, Guanrong CHEN

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

7 Citations (Scopus)


Evolutionary algorithms are cost-effective for solving real-world optimization problems, such as NP-hard and black-box problems. Before an evolutionary algorithm can be put into real-world applications, it is desirable that the algorithm was tested on a number of benchmark problems. On the other hand, performance measure on benchmarks can reflect if the benchmark suite is representative. In this paper, benchmarks are generated based on the performance comparison among a set of established algorithms. For each algorithm, its uniquely easy (or uniquely difficult) problem instances can be generated by an evolutionary algorithm. The unique difficulty nature of a problem instance to an algorithm is ensured by the Kruskal-Wallis H-test, assisted by a hierarchical fitness assignment method. Experimental results show that an algorithm performs the best (worst) consistently on its uniquely easy (difficult) problem. The testing results are repeatable. Some possible applications of this work include: 1) to compose an alternative benchmark suite; 2) to give a novel method for accessing novel algorithms; and 3) to generate a set of meaningful training and testing problems for evolutionary algorithm selectors and portfolios.

Original languageEnglish
Title of host publicationGECCO 2018 Companion - Proceedings of the 2018 Genetic and Evolutionary Computation Conference Companion
Editors Hernan AGUIRRE
Place of PublicationNew York, USA
PublisherAssociation for Computing Machinery, Inc
Number of pages5
ISBN (Electronic)9781450357647
Publication statusPublished - 6 Jul 2018
Externally publishedYes
Event2018 Genetic and Evolutionary Computation Conference, GECCO 2018 - Kyoto, Japan
Duration: 15 Jul 201819 Jul 2018


Conference2018 Genetic and Evolutionary Computation Conference, GECCO 2018

Bibliographical note

The work described in this paper was supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No. CityU 125313]. We thank Prof Hideyuki Takagi for the meaningful discussions and suggestions on the statistical issue.

Publisher Copyright:
© 2018 Association for Computing Machinery.


  • Algorithm performance measure
  • Evolutionary algorithm
  • Generating benchmark instance
  • Hierarchical fitness
  • Statistical test


Dive into the research topics of 'Evolving benchmark functions using kruskal-wallis test'. Together they form a unique fingerprint.

Cite this