Training error and sensitivity-based ensemble feature selection

Wing W. Y. NG; Yuxi TUO; Jianjun ZHANG; Sam KWONG

doi:10.1007/s13042-020-01120-8

Training error and sensitivity-based ensemble feature selection

Wing W. Y. NG, Yuxi TUO, Jianjun ZHANG, Sam KWONG

Research output: Journal Publications › Journal Article (refereed) › peer-review

10 Citations (Scopus)

Abstract

Ensemble feature selection combines feature selection and ensemble learning to improve the generalization capability of ensemble systems. However, current methods minimizing only the training error may not generalize well on future unseen samples. In this paper, we propose a training error and sensitivity-based ensemble feature selection method.The NSGA-III is applied to find optimal feature subsets by minimizing two objective functions of the whole ensemble system simultaneously: the training error and the sensitivity of the ensemble. With this scheme, the ensemble system maintains both high accuracy and high stability which is expected to achieve a high generalization capability. Experimental results on 18 datasets show that the proposed method significantly outperforms state-of-the-art methods.

Original language	English
Pages (from-to)	2313–2326
Journal	International Journal of Machine Learning and Cybernetics
Volume	11
Issue number	10
Early online date	13 Apr 2020
DOIs	https://doi.org/10.1007/s13042-020-01120-8
Publication status	Published - Oct 2020
Externally published	Yes

Bibliographical note

This work was supported by National Natural Science Foundation of China under Grants 61876066, 61572201 and 61672443, Guangdong Province Science and Technology Plan Project (Collaborative Innovation and Platform Environment Construction) 2019A050510006, Guangzhou Science and Technology Plan Project 201804010245, and Hong Kong RGC General Research Funds under Grant 9042038 (CityU 11205314) and Grant 9042322 (CityU 11200116).

Keywords

Ensemble
Feature selection
NSGA-III
Sensitivity

Access to Document

10.1007/s13042-020-01120-8

Cite this

@article{44ef96aaf25c460c9c23a67369905e28,

title = "Training error and sensitivity-based ensemble feature selection",

abstract = "Ensemble feature selection combines feature selection and ensemble learning to improve the generalization capability of ensemble systems. However, current methods minimizing only the training error may not generalize well on future unseen samples. In this paper, we propose a training error and sensitivity-based ensemble feature selection method.The NSGA-III is applied to find optimal feature subsets by minimizing two objective functions of the whole ensemble system simultaneously: the training error and the sensitivity of the ensemble. With this scheme, the ensemble system maintains both high accuracy and high stability which is expected to achieve a high generalization capability. Experimental results on 18 datasets show that the proposed method significantly outperforms state-of-the-art methods.",

keywords = "Ensemble, Feature selection, NSGA-III, Sensitivity",

author = "NG, {Wing W. Y.} and Yuxi TUO and Jianjun ZHANG and Sam KWONG",

note = "This work was supported by National Natural Science Foundation of China under Grants 61876066, 61572201 and 61672443, Guangdong Province Science and Technology Plan Project (Collaborative Innovation and Platform Environment Construction) 2019A050510006, Guangzhou Science and Technology Plan Project 201804010245, and Hong Kong RGC General Research Funds under Grant 9042038 (CityU 11205314) and Grant 9042322 (CityU 11200116).",

year = "2020",

month = oct,

doi = "10.1007/s13042-020-01120-8",

language = "English",

volume = "11",

pages = "2313–2326",

journal = "International Journal of Machine Learning and Cybernetics",

issn = "1868-8071",

publisher = "Springer Science + Business Media",

number = "10",

}

TY - JOUR

T1 - Training error and sensitivity-based ensemble feature selection

AU - NG, Wing W. Y.

AU - TUO, Yuxi

AU - ZHANG, Jianjun

AU - KWONG, Sam

N1 - This work was supported by National Natural Science Foundation of China under Grants 61876066, 61572201 and 61672443, Guangdong Province Science and Technology Plan Project (Collaborative Innovation and Platform Environment Construction) 2019A050510006, Guangzhou Science and Technology Plan Project 201804010245, and Hong Kong RGC General Research Funds under Grant 9042038 (CityU 11205314) and Grant 9042322 (CityU 11200116).

PY - 2020/10

Y1 - 2020/10

N2 - Ensemble feature selection combines feature selection and ensemble learning to improve the generalization capability of ensemble systems. However, current methods minimizing only the training error may not generalize well on future unseen samples. In this paper, we propose a training error and sensitivity-based ensemble feature selection method.The NSGA-III is applied to find optimal feature subsets by minimizing two objective functions of the whole ensemble system simultaneously: the training error and the sensitivity of the ensemble. With this scheme, the ensemble system maintains both high accuracy and high stability which is expected to achieve a high generalization capability. Experimental results on 18 datasets show that the proposed method significantly outperforms state-of-the-art methods.

AB - Ensemble feature selection combines feature selection and ensemble learning to improve the generalization capability of ensemble systems. However, current methods minimizing only the training error may not generalize well on future unseen samples. In this paper, we propose a training error and sensitivity-based ensemble feature selection method.The NSGA-III is applied to find optimal feature subsets by minimizing two objective functions of the whole ensemble system simultaneously: the training error and the sensitivity of the ensemble. With this scheme, the ensemble system maintains both high accuracy and high stability which is expected to achieve a high generalization capability. Experimental results on 18 datasets show that the proposed method significantly outperforms state-of-the-art methods.

KW - Ensemble

KW - Feature selection

KW - NSGA-III

KW - Sensitivity

UR - http://www.scopus.com/inward/record.url?scp=85083802122&partnerID=8YFLogxK

U2 - 10.1007/s13042-020-01120-8

DO - 10.1007/s13042-020-01120-8

M3 - Journal Article (refereed)

SN - 1868-8071

VL - 11

SP - 2313

EP - 2326

JO - International Journal of Machine Learning and Cybernetics

JF - International Journal of Machine Learning and Cybernetics

IS - 10

ER -

Training error and sensitivity-based ensemble feature selection

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this