Negative Correlation Learning (NCL) [CHECK END OF SENTENCE], [CHECK END OF SENTENCE] is a neural network ensemble learning algorithm which introduces a correlation penalty term to the cost function of each individual network so that each neural network minimizes its mean-square-error (MSE) together with the correlation. This paper describes NCL in detail and observes that the NCL corresponds to training the entire ensemble as a single learning machine that only minimizes the MSE without regularization. This insight explains that NCL is prone to overfitting the noise in the training set. The paper analyzes this problem and proposes the multiobjective regularized negative correlation learning (MRNCL) algorithm which incorporates an additional regularization term for the ensemble and uses the evolutionary multiobjective algorithm to design ensembles. In MRNCL, we define the crossover and mutation operators and adopt nondominated sorting algorithm with fitness sharing and rank-based fitness assignment. The experiments on synthetic data as well as real-world data sets demonstrate that MRNCL achieves better performance than NCL, especially when the noise level is nontrivial in the data set. In the experimental discussion, we give three reasons why our algorithm outperforms others. © 2006 IEEE.
|Number of pages
|IEEE Transactions on Knowledge and Data Engineering
|Early online date
|19 Feb 2010
|Published - Dec 2010
Bibliographical noteThis work is partially supported by a Dorothy Hodgkin Postgraduate Scholarship to the first author and an EPSRC grant (GR/T10671/01) to the second author. The authors also thank Dr. Yaochu Jin for providing the source code on regularizing neural networks using multiobjective evolutionary algorithms  for comparison with our algorithm.
- Multiobjective algorithm
- multiobjective learning
- negative correlation learning
- neural network ensembles
- neural networks