A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Hiu Tung WONG; Chi-Sing LEUNG; Sam KWONG

doi:10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Hiu Tung WONG, Chi-Sing LEUNG, Sam KWONG

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

1 Citation (Scopus)

Abstract

The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

Original language	English
Title of host publication	Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I
Editors	Derong LIU, Shengli XIE, Yuanqing LI, Dongbin ZHAO, El-Sayed M. EI-ALFY
Publisher	Springer, Cham
Pages	424-433
Number of pages	10
ISBN (Electronic)	9783319700878
ISBN (Print)	9783319700861
DOIs	https://doi.org/10.1007/978-3-319-70087-8_45
Publication status	Published - Nov 2017
Externally published	Yes
Event	24th International Conference on Neural Information Processing - Guangzhou, China Duration: 14 Nov 2017 → 18 Nov 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10634
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Neural Information Processing
Abbreviated title	ICONIP 2017
Country/Territory	China
City	Guangzhou
Period	14/11/17 → 18/11/17

Bibliographical note

The work was supported by a research grant from the Government of the Hong Kong Special Administrative Region (CityU 11259516).

Keywords

Additive noise
Feedforward networks
Multiplicative noise

Access to Document

10.1007/978-3-319-70087-8_45

Cite this

WONG, H. T., LEUNG, C.-S., & KWONG, S. (2017). A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. In D. LIU, S. XIE, Y. LI, D. ZHAO, & E.-S. M. EI-ALFY (Eds.), Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I (pp. 424-433). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634). Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_45

WONG, Hiu Tung ; LEUNG, Chi-Sing ; KWONG, Sam. / A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I. editor / Derong LIU ; Shengli XIE ; Yuanqing LI ; Dongbin ZHAO ; El-Sayed M. EI-ALFY. Springer, Cham, 2017. pp. 424-433 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{02007c9508cc416bb467776b1ebe79d7,

title = "A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks",

abstract = "The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.",

keywords = "Additive noise, Feedforward networks, Multiplicative noise",

author = "WONG, {Hiu Tung} and Chi-Sing LEUNG and Sam KWONG",

note = "The work was supported by a research grant from the Government of the Hong Kong Special Administrative Region (CityU 11259516).; 24th International Conference on Neural Information Processing, ICONIP 2017 ; Conference date: 14-11-2017 Through 18-11-2017",

year = "2017",

month = nov,

doi = "10.1007/978-3-319-70087-8_45",

language = "English",

isbn = "9783319700861",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Cham",

pages = "424--433",

editor = "Derong LIU and Shengli XIE and Yuanqing LI and Dongbin ZHAO and EI-ALFY, { El-Sayed M.}",

booktitle = "Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I",

}

WONG, HT, LEUNG, C-S & KWONG, S 2017, A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. in D LIU, S XIE, Y LI, D ZHAO & E-SM EI-ALFY (eds), Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10634, Springer, Cham, pp. 424-433, 24th International Conference on Neural Information Processing, Guangzhou, China, 14/11/17. https://doi.org/10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. / WONG, Hiu Tung; LEUNG, Chi-Sing; KWONG, Sam.
Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I. ed. / Derong LIU; Shengli XIE; Yuanqing LI; Dongbin ZHAO; El-Sayed M. EI-ALFY. Springer, Cham, 2017. p. 424-433 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634).

Research output: Book Chapters | Papers in Conference Proceedings › Conference paper (refereed) › Research › peer-review

TY - GEN

T1 - A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

AU - WONG, Hiu Tung

AU - LEUNG, Chi-Sing

AU - KWONG, Sam

N1 - The work was supported by a research grant from the Government of the Hong Kong Special Administrative Region (CityU 11259516).

PY - 2017/11

Y1 - 2017/11

N2 - The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

AB - The incremental extreme learning machine (I-ELM) algorithm provides a low computational complexity training mechanism for single-hidden layer feedforward neworks (SLFNs). However, the original I-ELM algorithm does not consider the node noise situation, and node noise may greatly degrade the performance of a trained SLFN. This paper presents a generalized node noise resistant I-ELM (GNNR-I-ELM) for SLFNs. We first define a noise resistant training objective function for SLFNs. Afterwards, we develop the GNNR-I-ELM algorithm which adds τ nodes into the network at each iteration. The GNNR-I-ELM algorithm estimates the output weights of the newly additive nodes and does not change all the previously trained output weights. Its noise tolerant ability is much better than that of the original I-ELM. Besides, we prove that in terms of the training set mean squared error of noisy networks, the GNNR-I-ELM algorithm converges.

KW - Additive noise

KW - Feedforward networks

KW - Multiplicative noise

UR - http://www.scopus.com/inward/record.url?scp=85035088364&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-70087-8_45

DO - 10.1007/978-3-319-70087-8_45

M3 - Conference paper (refereed)

SN - 9783319700861

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 424

EP - 433

BT - Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I

A2 - LIU, Derong

A2 - XIE, Shengli

A2 - LI, Yuanqing

A2 - ZHAO, Dongbin

A2 - EI-ALFY, El-Sayed M.

PB - Springer, Cham

T2 - 24th International Conference on Neural Information Processing

Y2 - 14 November 2017 through 18 November 2017

ER -

WONG HT, LEUNG CS, KWONG S. A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks. In LIU D, XIE S, LI Y, ZHAO D, EI-ALFY ESM, editors, Neural Information Processing: 24th International Conference, ICONIP 2017, Guangzhou, China, November 14-18, 2017, Proceedings, Part I. Springer, Cham. 2017. p. 424-433. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-70087-8_45

A generalized I-ELM algorithm for handling node noise in single-hidden layer feedforward networks

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this