Multiobjective neural network ensembles based on regularized negative correlation learning

Huanhuan CHEN, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

125 Citations (Scopus)

Abstract

Negative Correlation Learning (NCL) [CHECK END OF SENTENCE], [CHECK END OF SENTENCE] is a neural network ensemble learning algorithm which introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean-square-error (MSE) together with the correlation. This paper describes NCL in detail and observes that the NCL corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This insight explains that NCL is prone to overfitting the noise in the training set. The paper analyzes this problem and proposes the multiobjective regularized negative correlation learning (MRNCL) algorithm which incorporates an additional regularization term for the ensemble and uses the evolutionary multiobjective algorithm to design ensembles. In MRNCL, we define the crossover and mutation operators and adopt nondominated sorting algorithm with fitness sharing and rank-based fitness assignment. The experiments on synthetic data as well as real-world data sets demonstrate that MRNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. In the experimental discussion, we give three reasons why our algorithm outperforms others. © 2006 IEEE.
Original languageEnglish
Article number5416712
Pages (from-to)1738-1751
Number of pages14
JournalIEEE Transactions on Knowledge and Data Engineering
Volume22
Issue number12
Early online date19 Feb 2010
DOIs
Publication statusPublished - Dec 2010
Externally publishedYes

Bibliographical note

This work is partially supported by a Dorothy Hodgkin Postgraduate Scholarship to the first author and an EPSRC grant (GR/T10671/01) to the second author. The authors also thank Dr. Yaochu Jin for providing the source code on regularizing neural networks using multiobjective evolutionary algorithms [12] for comparison with our algorithm.

Keywords

  • Multiobjective algorithm
  • multiobjective learning
  • negative correlation learning
  • neural network ensembles
  • neural networks
  • regularization

Fingerprint

Dive into the research topics of 'Multiobjective neural network ensembles based on regularized negative correlation learning'. Together they form a unique fingerprint.

Cite this