Bagging and boosting negatively correlated neural networks

Md. Monirul ISLAM, Xin YAO, S.M. Shahriar NIRJON, Muhammad Asiful ISLAM, Kazuyuki MURASE

Research output: Journal PublicationsJournal Article (refereed)peer-review

74 Citations (Scopus)

Abstract

In this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization. © 2008 IEEE.
Original languageEnglish
Pages (from-to)771-784
Number of pages14
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume38
Issue number3
Early online date21 May 2008
DOIs
Publication statusPublished - Jun 2008
Externally publishedYes

Bibliographical note

The work of X. Yao was supported in part by the Engineering and Physical Sciences Research Council (U.K.) under Grant GR/T10671/01 and by the Fund for Foreign Scholars in University Research and Teaching Programs (China) under Grant B07033. This paper was recommended by Associate Editor N. Chawla.

Keywords

  • Bagging
  • Boosting
  • Constructive approach
  • Diversity
  • Generalization
  • Negative correlation learning
  • Neural network (NN) ensemble design

Fingerprint

Dive into the research topics of 'Bagging and boosting negatively correlated neural networks'. Together they form a unique fingerprint.

Cite this