Reusing genetic programming for ensemble selection in classification of unbalanced data

Urvesh BHOWAN, Mark JOHNSTON, Mengjie ZHANG, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

78 Citations (Scopus)

Abstract

Classification algorithms can suffer from performance degradation when the class distribution is unbalanced. This paper develops a two-step approach to evolving ensembles using genetic programming (GP) for unbalanced data. The first step uses multiobjective (MO) GP to evolve a Pareto-approximated front of GP classifiers to form the ensemble by trading-off the minority and the majority class against each other during learning. The MO component alleviates the reliance on sampling to artificially rebalance the data. The second step, which is the focus this paper, proposes a novel ensemble selection approach using GP to automatically find/choose the best individuals for the ensemble. This new GP approach combines multiple Pareto-approximated front members into a single composite genetic program solution to represent the (optimized) ensemble. This ensemble representation has two main advantages/novelties over traditional genetic algorithm (GA) approaches. First, by limiting the depth of the composite solution trees, we use selection pressure during evolution to find small highly-cooperative groups of individuals for the ensemble. This means that ensemble sizes are not fixed a priori (as in GA), but vary depending on the strength of the base learners. Second, we compare different function set operators in the composite solution trees to explore new ways to aggregate the member outputs and thus, control how the ensemble computes its output. We show that the proposed GP approach evolves smaller more diverse ensembles compared to an established ensemble selection algorithm, while still performing as well as, or better than the established approach. The evolved GP ensembles also perform well compared to other bagging and boosting approaches, particularly on tasks with high levels of class imbalance. © 1997-2012 IEEE.
Original languageEnglish
Article number6677603
Pages (from-to)893-908
Number of pages16
JournalIEEE Transactions on Evolutionary Computation
Volume18
Issue number6
Early online date31 Jan 2014
DOIs
Publication statusPublished - Dec 2014
Externally publishedYes

Funding

This work is supported in part by the Marsden Fund of the New Zealand Government under Contract VUW0806, administrated by the Royal Society of New Zealand, in part by an NSFC Grant 61329302, in part by the European Union Seventh Framework Programme Grant 270428, and in part by a Royal Society Wolfson Research Merit Award.

Keywords

  • Classification
  • ensemble machine learning
  • genetic programming
  • unbalanced data

Fingerprint

Dive into the research topics of 'Reusing genetic programming for ensemble selection in classification of unbalanced data'. Together they form a unique fingerprint.

Cite this