Mixture of Channel Experts Boosts CNN Efficiency

Mixture-of-Experts (MoE) architectures have successfully scaled language models, but directly applying this design to convolutional networks by duplicating parallel convolutional experts often leads to redundant filters. This research proposes a new approach called Mixture of Channel Experts (MoCE), which redefines the expert axis to focus on channel selection rather than operator duplication. MoCE is a structured sparse channel-mixing layer that replaces traditional pointwise (1x1) channel-reduction projections. In this design, each "expert" is a single output channel with a learned sparse support of a small number of input channels. An input-adaptive softmax predicts how these selected channels are combined, allowing each expert to dynamically shift between mean-like and max-like aggregation. A residual expert handles unselected channels, and a load-balancing loss ensures comprehensive channel coverage. This method reduces the quadratic computational cost of dense projections to a cost scaling as k/C (where k is sparse support size and C is total channels), translating to measured wall-clock time savings. Evaluations on ResNet backbones across ImageNet-1K and CIFAR-100, transfer learning, and EfficientViT show MoCE matches or surpasses dense baselines and prior channel-selection methods, while reducing MACs by 16.7% and end-to-end latency.