Towards Robust Uncertainty Estimation in the Presence of Noisy Labels

Chao PAN, Bo YUAN, Wei ZHOU, Xin YAO

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

1 Citation (Scopus)

Abstract

In security-critical applications, it is essential to know how confident the model is in its predictions. Many uncertainty estimation methods have been proposed recently, and these methods are reliable when the training data do not contain labeling errors. However, we find that the quality of these uncertainty estimation methods decreases dramatically when noisy labels are present in the training data. In some datasets, the uncertainty estimates would become completely absurd, even though these labeling noises barely affect the test accuracy. We further analyze the impact of existing label noise handling methods on the reliability of uncertainty estimates, although most of these methods focus only on improving the accuracy of the models. We identify that the data cleaning-based approach can alleviate the influence of label noise on uncertainty estimates to some extent, but there are still some drawbacks. Finally, we propose a robust uncertainty estimation method under label noise. Compared with other algorithms, our approach achieves a more reliable uncertainty estimates in the presence of noisy labels, especially when there are large-scale labeling errors in the training data. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
Original languageEnglish
Title of host publicationArtificial Neural Networks and Machine Learning - ICANN 2022 : 31st International Conference on Artificial Neural Networks, Bristol, UK, September 6–9, 2022, Proceedings, Part I
EditorsElias PIMENIDIS, Plamen ANGELOV, Chrisina JAYNE, Antonios PAPALEONIDAS, Mehmet AYDIN
PublisherSpringer Science and Business Media Deutschland GmbH
Pages673-684
Number of pages12
ISBN (Electronic)9783031159190
ISBN (Print)9783031159183
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event31st International Conference on Artificial Neural Networks - Bristol, United Kingdom
Duration: 6 Sept 20229 Sept 2022

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume13529
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference31st International Conference on Artificial Neural Networks
Abbreviated titleICANN 2022
Country/TerritoryUnited Kingdom
CityBristol
Period6/09/229/09/22

Bibliographical note

This work was supported by the Research Institute of Trustworthy Autonomous Systems, the Guangdong Provincial Key Laboratory (Grant No. 2020B121201001), the Program for Guangdong Introducing Innovative and Entrepreneurial Teams (Grant No. 2017ZT07X386), the Shenzhen Science and Technology Program (Grant No. KQTD2016112514355531) and Huawei project on “Fundamental Theory and Key Technologies of Trustworthy Systems”.

Keywords

  • Mis-classification detection
  • Noisy label
  • Out-of-distribution data
  • Uncertainty estimation

Fingerprint

Dive into the research topics of 'Towards Robust Uncertainty Estimation in the Presence of Noisy Labels'. Together they form a unique fingerprint.

Cite this