Online Learning in Open Data Space

In real-world data stream mining, instances are typically distributed in open data space where the composition of classes and features undergoes unpredictable changes, leading to the dual challenges of class evolution and feature evolution. Although several early studies simultaneously address both issues, they necessitate substantial data storage for model adaptation. Numerous subsequent studies focus on the development of online learning algorithms. However, these algorithms tackle the evolution in either feature space or label space, not both. In this paper, we propose a novel Ensemble of Passive-Aggressive Model (EPAM) to address challenges associated with the open data space in an online learning scenario. We initiate the research of online learning in the open data space by constructing several baseline models grounded in state-of-the-art online learning methods in the domains of class evolution and feature evolution, followed by an analysis of their limitations in handling real-world data streams. To overcome these limitations, EPAM incorporates a novel feature contributed bias classifier specifically designed for feature evolution, with the bias term capable of adapting to diverse feature spaces. Furthermore, a novel model adaptation strategy is developed to balance error feedback among classes designated as the negative class for each feature contributed bias classifier, therefore addressing the dynamic class imbalance induced by class evolution and enhancing multi-class classification performance. Comprehensive experiments on various synthetic and real-world data streams demonstrate the superior performance of EPAM.