Many-objective test suite generation for software product lines

Robert M. HIERONS, Miqing LI, Xiaohui LIU, Jose Antonio PAREJO, Sergio SEGURA, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

19 Citations (Scopus)


A Software Product Line (SPL) is a set of products built from a number of features, the set of valid products being defined by a feature model. Typically, it does not make sense to test all products defined by an SPL and one instead chooses a set of products to test (test selection) and, ideally, derives a good order in which to test them (test prioritisation). Since one cannot know in advance which products will reveal faults, test selection and prioritisation are normally based on objective functions that are known to relate to likely effectiveness or cost. This article introduces a new technique, the grid-based evolution strategy (GrES), which considers several objective functions that assess a selection or prioritisation and aims to optimise on all of these. The problem is thus a many-objective optimisation problem.We use a new approach, in which all of the objective functions are considered but one (pairwise coverage) is seen as the most important. We also derive a novel evolution strategy based on domain knowledge. The results of the evaluation, on randomly generated and realistic feature models, were promising, with GrES outperforming previously proposed techniques and a range of many-objective optimisation algorithms. © 2020 Association for Computing Machinery.
Original languageEnglish
Article number3361146
JournalACM Transactions on Software Engineering and Methodology
Issue number1
Early online date30 Jan 2020
Publication statusPublished - 31 Jan 2020
Externally publishedYes

Bibliographical note

This work has been partially supported by the European Commission (FEDER) and Spanish Government under CICYT projects BELI (TIN2015-70560-R) and HORATIO (RTI2018-101204-B-C21), the Science and Technology Innovation Committee Foundation of Shenzhen (ZDSYS201703031748284), Shenzhen Peacock Plan (KQTD2016112514355531), the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (Grant No. 2017ZT07X386), and EPSRC (EP/J017515/1 and EP/P005578/1).


  • Multi-objective optimisation
  • Software product line
  • Test prioritisation
  • Test selection


Dive into the research topics of 'Many-objective test suite generation for software product lines'. Together they form a unique fingerprint.

Cite this