Adaptive Incremental Provenance Analysis for Trustworthy Federated Learning

Federated learning allows multiple parties to collaboratively train AI models without sharing raw data—but this privacy advantage comes with risks. A major threat is poisoning attacks, where malicious participants submit fake model updates to sabotage the global model. Current defenses typically use statistical methods to filter out suspicious updates, but these approaches struggle to detect subtle attacks in real time and offer little insight into why an update was flagged. To address this, we introduce Incremental Provenance Analysis (IPA), a new defense framework that monitors how model updates evolve over time ΔW. Unlike traditional methods, IPA doesn’t just look for outliers—it learns the typical “provenance” of benign updates. When an update deviates suspiciously, IPA not only detects it but also traces the attack source and adjusts the aggregation strategy dynamically. Our experiments show IPA effectively thwarts poisoning attempts while maintaining model accuracy, offering a more transparent and adaptive solution for secure federated learning.