Federated Learning Aggregation Robustness Under Poisoning and Backdoor Attacks

Federated learning, a distributed machine learning approach, faces significant security challenges, particularly from model poisoning and backdoor attacks. This research undertakes a comprehensive reconstruction and analysis of a 500-cell benchmark matrix to evaluate the robustness of different federated aggregation methods. The study considers five aggregation techniques, five datasets, five architectures, and four attack conditions: clean, sign-flipping, Gaussian, and BadNets. The analysis, based on 454 original and 36 repaired/rerun executions, revealed distinct performance profiles for the aggregation methods. Trimmed Mean demonstrated the highest macro-mean accuracy (76.02%) in clean conditions and the lowest mean within-task rank. Conversely, Krum proved most effective against both sign-flipping and Gaussian attacks, achieving the highest recorded accuracy under these adversarial configurations. These relative rankings remained consistent even when the analysis was restricted to tasks with complete original logs. Crucially, the researchers also identified critical issues with the supplied metric implementations. An audit of the BadNets metric revealed that it measures "Triggered Target-Label Rate" (TTLR) rather than a conventional attack success rate that excludes the target label. Furthermore, a potential discrepancy was found in the FedPARETO scaffold, where predictive summaries might reflect an uncorrupted local model while a separately corrupted update is applied for aggregation. The study emphasizes that its findings are descriptive comparisons within the recorded configurations due to the single seed per cell and incomplete attack lineage.