Flower Hub Platform Enhances Federated Learning Benchmarking

Researchers have launched Flower Hub, a novel platform aimed at standardizing and streamlining the benchmarking process for federated learning (FL) applications. The platform addresses common challenges in FL evaluation, such as reproducibility issues, difficulties in comparing results across different studies, and limited extensibility of existing benchmarks. Flower Hub achieves this by packaging benchmarks as self-contained, executable, and versioned applications, complete with standardized metadata, pinned dependencies, and explicit evaluation workflows. The platform includes a multi-domain benchmark suite covering diverse settings like cross-silo and cross-device FL, with tasks ranging from medical imaging and financial tabular learning to legal instruction tuning and phishing URL detection. A key feature is its ability to run the same benchmarking application seamlessly across both simulated and deployed environments without requiring code changes, ensuring consistent evaluation. Beyond model quality, Flower Hub also supports system-aware reporting, capturing crucial metrics like runtime and communication overhead. This initiative marks a significant step towards more portable, executable, and reusable FL benchmarks.