Simultaneous training of negatively correlated neural networks in an ensemble

Yong LIU, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

245 Citations (Scopus)

Abstract

This paper presents a new cooperative ensemble learning system (CELS) for designing neural network ensembles. The idea behind CELS is to encourage different individual networks in an ensemble to learn different parts or aspects of a training data so that the ensemble can learn the whole training data better. In CELS, the individual networks are trained simultaneously rather than independently or sequentially. This provides an opportunity for the individual networks to interact with each other and to specialize. CELS can create negatively correlated neural networks using a correlation penalty term in the error function to encourage such specialization. This paper analyzes CELS in terms of bias-variance-covariance tradeoff. CELS has also been tested on the Mackey-Glass time series prediction problem and the Australian credit card assessment problem. The experimental results show that CELS can produce neural network ensembles with good generalization ability. © 1999 IEEE.
Original languageEnglish
Pages (from-to)716-725
Number of pages10
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume29
Issue number6
DOIs
Publication statusPublished - 1999
Externally publishedYes

Keywords

  • Generalization
  • Negative correlation learning
  • Neural network ensembles

Fingerprint

Dive into the research topics of 'Simultaneous training of negatively correlated neural networks in an ensemble'. Together they form a unique fingerprint.

Cite this