Many real-world data stream applications suffer from verification latency, where the labels of the training examples arrive with a delay. In binary classification problems, the labeling process frequently involves waiting for a pre-determined period of time to observe an event that assigns the example to a given class. Once this time passes, if such labeling event does not occur, the example is labeled as belonging to the other class. For example, in software defect prediction, one may wait to see if a defect is associated to a software change implemented by a developer, producing a defect-inducing training example. If no defect is found during the waiting time, the training example is labeled as clean. Such verification latency inherently causes label noise associated to insufficient waiting time. For example, a defect may be observed only after the pre-defined waiting time has passed, resulting in a noisy example of the clean class. Due to the nature of the waiting time, such noise is frequently one-sided, meaning that it only occurs to examples of one of the classes. However, no existing work tackles label noise associated to verification latency. This paper proposes a novel data stream learning approach that estimates the confidence in the labels assigned to the training examples and uses this to improve predictive performance in problems with one-sided label noise. Our experiments with 14 real-world datasets from the domain of software defect prediction demonstrate the effectiveness of the proposed approach compared to existing ones. © 2022 IEEE.
|Title of host publication
|Proceedings of the International Joint Conference on Neural Networks
|Institute of Electrical and Electronics Engineers Inc.
|Published - 18 Jul 2022
Bibliographical noteThis work was supported by National Natural Science Foundation of China (Grant No. 62002148), EPSRC (Grant No. EP/R006660/2), Guangdong Provincial Key Laboratory (Grant No. 2020B121201001), the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (Grant No. 2017ZT07X386), Shenzhen Science and Technology Program (Grant No. KQTD2016112514355531), Research Institute of Trustworthy Autonomous Systems (RITAS), and a joint project between Huawei and Southern University of Science and Technology.
- concept drift
- confidence level
- Data stream learning
- just-in-time software defect prediction
- one-sided label noise
- verification latency