Quality evaluation of solution sets in multiobjective optimisation: A survey

Miqing LI, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

255 Citations (Scopus)


Complexity and variety of modern multiobjective optimisation problems result in the emergence of numerous search techniques, from traditional mathematical programming to various randomised heuristics. A key issue raised consequently is how to evaluate and compare solution sets generated by these multiobjective search techniques. In this article, we provide a comprehensive review of solution set quality evaluation. Starting with an introduction of basic principles and concepts of set quality evaluation, this article summarises and categorises 100 state-of-the-art quality indicators, with the focus on what quality aspects these indicators reflect. This is accompanied in each category by detailed descriptions of several representative indicators and in-depth analyses of their strengths and weaknesses. Furthermore, issues regarding attributes that indicators possess and properties that indicators are desirable to have are discussed, in the hope of motivating researchers to look into these important issues when designing quality indicators and of encouraging practitioners to bear these issues in mind when selecting/using quality indicators. Finally, future trends and potential research directions in the area are suggested, together with some guidelines on these directions. © 2019 Association for Computing Machinery.
Original languageEnglish
Article numbera26
JournalACM Computing Surveys
Issue number2
Early online date18 Mar 2019
Publication statusPublished - 31 Mar 2020
Externally publishedYes

Bibliographical note

This work was supported by National Key R&D Program of China (Grant No. 2017YFC0804003), EPSRC (Grants No. EP/J017515/1 and No. EP/P005578/1), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (Grant No. 2017ZT07X386), Shenzhen Peacock Plan (Grant No. KQTD2016112514355531), the Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. ZDSYS201703031748284), and the Program for University Key Laboratory of Guangdong Province (Grant No. 2017KSYS008).


  • Evolutionary algorithms
  • Exact method
  • Heuristic
  • Indicator
  • Measure
  • Metaheuristic
  • Metric
  • Multi-criteria optimisation
  • Multobjective optimisation
  • Performance assessment
  • Quality evaluation


Dive into the research topics of 'Quality evaluation of solution sets in multiobjective optimisation: A survey'. Together they form a unique fingerprint.

Cite this