Learnijoy
ResearchAI ResearchAI Engineering & DevTools

FedFMX Boosts Federated Class-Incremental Learning with Expert Routing.

Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Zewei Liu, Edith Cheuk Han Ngai· June 30, 2026 View original

Summary

This paper introduces FedFMX, a novel framework for Federated Class-Incremental Learning that uses Fisher-Routed Mixture of Experts to address capacity conflict, catastrophic forgetting, data heterogeneity, and class misalignment. It adaptively specializes experts across clients, achieving superior performance on benchmarks.

Federated Learning (FL) is a promising distributed machine learning paradigm, but it faces significant hurdles when applied to class-incremental learning scenarios. These challenges include the shared model becoming overloaded and suffering from catastrophic forgetting, the inherent heterogeneity of non-IID data across clients, and issues with synchronized class alignment. To tackle these problems, researchers propose FedFMX (Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning). This framework enables adaptive expert specialization among different clients. The core idea is to route each data sample to a specific subset of experts, optimizing both the acquisition of new knowledge and the retention of existing knowledge. FedFMX incorporates a Fisher-Routed Expert Scoring (FRES) module to assess expert importance based on stability cost and plasticity gain. An Adaptive Expert Selection (AES) module then quantifies marginal contributions to determine the optimal expert subset. Finally, routing-aware regularization ensures load balance and efficient FL training. Theoretical analysis confirms an O(T^-1) convergence rate, and extensive experiments show FedFMX outperforms state-of-the-art methods.

Why it matters

For organizations implementing federated learning, especially in dynamic environments where new data classes emerge over time, FedFMX offers a robust solution to maintain model performance, prevent forgetting, and handle diverse client data without compromising privacy.

How to implement this in your domain

  1. 1Assess existing federated learning pipelines for challenges related to class-incremental learning or data heterogeneity.
  2. 2Investigate the architecture of Mixture of Experts (MoE) models and their applicability in FL settings.
  3. 3Explore methods for adaptive expert routing and load balancing in distributed machine learning.
  4. 4Consider integrating Fisher information-based metrics for evaluating model stability and plasticity.
  5. 5Benchmark FedFMX or similar MoE-based FL approaches against current incremental learning solutions.

Who benefits

HealthcareFinanceTelecommunicationsIoTSmart Cities

Key takeaways

  • FedFMX addresses key challenges in federated class-incremental learning.
  • It uses adaptive expert specialization and intelligent sample routing.
  • Fisher-Routed Expert Scoring balances knowledge acquisition and retention.
  • The framework achieves superior performance and theoretical convergence guarantees.

Original post by Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Zewei Liu, Edith Cheuk Han Ngai

"arXiv:2606.28835v1 Announce Type: new Abstract: Federated Learning (FL) emerged as a promising distributed machine learning paradigm. However, extending FL to the class incremental learning scenarios introduces unique challenges: 1) Capacity conflict and catastrophic forgetting f…"

View on X

Originally posted by Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Zewei Liu, Edith Cheuk Han Ngai on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Research

AI ResearchAI Engineering & DevTools

BaRA Improves LoRA Fine-Tuning with Adaptive Rank Allocation

Researchers introduce BaRA, a Bayesian Adaptive Rank Allocation framework for parameter-efficient fine-tuning, which dynamically adjusts adaptation capacity based on context. This method enhances predictive performance, robustness, and uncertainty calibration compared to standard LoRA and other Bayesian LoRA variants.

Zhibin Duan, Yuhong Wang, Jiahong Fu, Zongsheng Yue, Bo Chen, Zongben XuJun 30, 2026
AI ResearchAI Engineering & DevTools

New Preconditioner Improves Deep Network Training Stability and Performance

Researchers introduce Dead-Direction Conditioners (DDC), a novel preconditioning method that leverages gauge-equivariant optimization to prevent deep network training from drifting along symmetry orbits. This technique improves model stability, reduces overfitting, and enhances performance in language and vision models.

Tejas Pradeep ShirodkarJun 30, 2026
AI ResearchAI Engineering & DevTools

SMDA Traces Training Data Influence on LLM Behavioral Policies

Researchers introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes specific training examples to the interpretable symbolic policies governing an LLM's high-level behavior. SMDA offers a fine-grained diagnostic tool to understand how training data shapes model decisions, revealing safety gaps and unintended influences.

Reza Habibi, Darian Lee, Magy Seif El-NasrJun 30, 2026