A new adaptive merging and growing algorithm for designing artificial neural networks

Md. Monirul ISLAM, Md. Abdus SATTAR, Md. Faijul AMIN, Xin YAO, Kazuyuki MURASE

Research output: Journal PublicationsJournal Article (refereed)peer-review

111 Citations (Scopus)

Abstract

This paper presents a new algorithm, called adaptive merging and growing algorithm (AMGA), in designing artificial neural networks (ANNs). This algorithm merges and adds hidden neurons during the training process of ANNs. The merge operation introduced in AMGA is a kind of a mixed mode operation, which is equivalent to pruning two neurons and adding one neuron. Unlike most previous studies, AMGA puts emphasis on autonomous functioning in the design process of ANNs. This is the main reason why AMGA uses an adaptive not a predefined fixed strategy in designing ANNs. The adaptive strategy merges or adds hidden neurons based on the learning ability of hidden neurons or the training progress of ANNs. In order to reduce the amount of retraining after modifying ANN architectures, AMGA prunes hidden neurons by merging correlated hidden neurons and adds hidden neurons by splitting existing hidden neurons. The proposed AMGA has been tested on a number of benchmark problems in machine learning and ANNs, including breast cancer, Australian credit card assessment, and diabetes, gene, glass, heart, iris, and thyroid problems. The experimental results show that AMGA can design compact ANN architectures with good generalization ability compared to other algorithms. © 2009 IEEE.
Original languageEnglish
Pages (from-to)705-722
Number of pages18
JournalIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Volume39
Issue number3
Early online date13 Feb 2009
DOIs
Publication statusPublished - Jun 2009
Externally publishedYes

Bibliographical note

This work was supported in part by the Japanese Society for Promotion of Science (JSPS), by the Yazaki Memorial Foundation for Science and Technology, and by the University of Fukui through grants given to K. Murase. The work of Md. M. Islam was supported by the JSPS through a fellowship. This paper was recommended by Associate Editor S. Hu.

Keywords

  • Adding neurons
  • Artificial neural network (ANN) design
  • Generalization ability
  • Merging neurons
  • Retraining

Fingerprint

Dive into the research topics of 'A new adaptive merging and growing algorithm for designing artificial neural networks'. Together they form a unique fingerprint.

Cite this