A constructive algorithm for training cooperative neural network ensembles

Md. Monirul ISLAM, Xin YAO, Kazuyuki MURASE

Research output: Journal PublicationsJournal Article (refereed)peer-review

265 Citations (Scopus)

Abstract

This paper presents a constructive algorithm for training cooperative neural-network ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on both accuracy and diversity among individual NNs in an ensemble. In order to maintain accuracy among individual NNs, the number of hidden nodes in individual NNs are also determined by a constructive approach. Incremental training based on negative correlation is used in CNNE to train individual NNs for different numbers of training epochs. The use of negative correlation learning and different training epochs for training individual NNs reflect CNNEs emphasis on diversity among individual NNs in an ensemble. CNNE has been tested extensively on a number of benchmark problems in machine learning and neural networks, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, soybean, and Mackey-Glass time series prediction problems. The experimental results show that CNNE can produce NN ensembles with good generalization ability.
Original languageEnglish
Pages (from-to)820-834
Number of pages15
JournalIEEE Transactions on Neural Networks
Volume14
Issue number4
DOIs
Publication statusPublished - Jul 2003
Externally publishedYes

Keywords

  • Constructive approach
  • Diversity
  • Generalization
  • Negative correlation learning
  • Neural-network (NN) ensemble design

Fingerprint

Dive into the research topics of 'A constructive algorithm for training cooperative neural network ensembles'. Together they form a unique fingerprint.

Cite this