Evolutionary algorithms are cost-effective for solving real-world optimization problems, such as NP-hard and black-box problems. Before an evolutionary algorithm can be put into real-world applications, it is desirable that the algorithm was tested on a number of benchmark problems. On the other hand, performance measure on benchmarks can reflect if the benchmark suite is representative. In this paper, benchmarks are generated based on the performance comparison among a set of established algorithms. For each algorithm, its uniquely easy (or uniquely difficult) problem instances can be generated by an evolutionary algorithm. The unique difficulty nature of a problem instance to an algorithm is ensured by the Kruskal-Wallis H-test, assisted by a hierarchical fitness assignment method. Experimental results show that an algorithm performs the best (worst) consistently on its uniquely easy (difficult) problem. The testing results are repeatable. Some possible applications of this work include: 1) to compose an alternative benchmark suite; 2) to give a novel method for accessing novel algorithms; and 3) to generate a set of meaningful training and testing problems for evolutionary algorithm selectors and portfolios.
|Title of host publication||GECCO 2018 Companion - Proceedings of the 2018 Genetic and Evolutionary Computation Conference Companion|
|Place of Publication||New York, USA|
|Publisher||Association for Computing Machinery, Inc|
|Number of pages||5|
|Publication status||Published - 6 Jul 2018|
|Event||2018 Genetic and Evolutionary Computation Conference, GECCO 2018 - Kyoto, Japan|
Duration: 15 Jul 2018 → 19 Jul 2018
|Conference||2018 Genetic and Evolutionary Computation Conference, GECCO 2018|
|Period||15/07/18 → 19/07/18|
Bibliographical noteThe work described in this paper was supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China [Project No. CityU 125313]. We thank Prof Hideyuki Takagi for the meaningful discussions and suggestions on the statistical issue.
© 2018 Association for Computing Machinery.
- Algorithm performance measure
- Evolutionary algorithm
- Generating benchmark instance
- Hierarchical fitness
- Statistical test