Software effort interval prediction via Bayesian inference and synthetic bootstrap resampling

Liyan SONG, Leandro L. MINKU, Y.A.O. XIN

Research output: Journal PublicationsJournal Article (refereed)peer-review

21 Citations (Scopus)

Abstract

Software effort estimation (SEE) usually suffers from inherent uncertainty arising from predictive model limitations and data noise. Relying on point estimation only may ignore the uncertain factors and lead project managers (PMs) to wrong decision making. Prediction intervals (PIs) with confidence levels (CLs) present a more reasonable representation of reality, potentially helping PMs to make better-informed decisions and enable more flexibility in these decisions. However, existing methods for PIs either have strong limitations or are unable to provide informative PIs. To develop a “better” effort predictor, we propose a novel PI estimator called Synthetic Bootstrap ensemble of Relevance Vector Machines (SynB-RVM) that adopts Bootstrap resampling to produce multiple RVM models based on modified training bags whose replicated data projects are replaced by their synthetic counterparts. We then provide three ways to assemble those RVM models into a final probabilistic effort predictor, from which PIs with CLs can be generated. When used as a point estimator, SynB-RVM can either significantly outperform or have similar performance compared with other investigated methods. When used as an uncertain predictor, SynB-RVM can achieve significantly narrower PIs compared to its base learner RVM. Its hit rates and relative widths are no worse than the other compared methods that can provide uncertain estimation. © 2019 Association for Computing Machinery.
Original languageEnglish
Article number5
JournalACM Transactions on Software Engineering and Methodology
Volume28
Issue number1
Early online date9 Jan 2019
DOIs
Publication statusPublished - 31 Jan 2019
Externally publishedYes

Bibliographical note

This work was supported by National Key R&D Program of China (Grant No. 2017YFC0804003), EPSRC (Grant Nos. EP/J017515/1, EP/R006660/1, and EP/P005578/1), the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (Grant No. 2017ZT07X386), Shenzhen Peacock Plan (Grant No. KQTD2016112514355531), the Science and Technology Innovation Committee Foundation of Shenzhen (Grant No. ZDSYS201703031748284), and the Program for University Key Laboratory of Guangdong Province (Grant No. 2017KSYS008).

Keywords

  • Bootstrap resampling
  • Ensemble learning
  • Prediction intervals with confidence levels
  • Relevance vector machine
  • Software effort estimation
  • Software risk management
  • Synthetic replacement
  • Uncertain effort estimation

Fingerprint

Dive into the research topics of 'Software effort interval prediction via Bayesian inference and synthetic bootstrap resampling'. Together they form a unique fingerprint.

Cite this