The impact of diversity on online ensemble learning in the presence of concept drift

Leandro L. MINKU, Allan P. WHITE, Xin YAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

358 Citations (Scopus)

Abstract

Online learning algorithms often have to operate in the presence of concept drift (i.e., the concepts to be learned can change with time). This paper presents a new categorization for concept drift, separating drifts according to different criteria into mutually exclusive and nonheterogeneous categories. Moreover, although ensembles of learning machines have been used to learn in the presence of concept drift, there has been no deep study of why they can be helpful for that and which of their features can contribute or not for that. As diversity is one of these features, we present a diversity analysis in the presence of different types of drifts. We show that, before the drift, ensembles with less diversity obtain lower test errors. On the other hand, it is a good strategy to maintain highly diverse ensembles to obtain lower test errors shortly after the drift independent on the type of drift, even though high diversity is more important for more severe drifts. Longer after the drift, high diversity becomes less important. Diversity by itself can help to reduce the initial increase in error caused by a drift, but does not provide the faster recovery from drifts in long-term. © 2006 IEEE.
Original languageEnglish
Article number5156502
Pages (from-to)730-742
Number of pages13
JournalIEEE Transactions on Knowledge and Data Engineering
Volume22
Issue number5
Early online date7 Jul 2009
DOIs
Publication statusPublished - May 2010
Externally publishedYes

Keywords

  • Concept drift
  • Diversity
  • Neural network ensembles
  • Online learning

Fingerprint

Dive into the research topics of 'The impact of diversity on online ensemble learning in the presence of concept drift'. Together they form a unique fingerprint.

Cite this