3D Masked Autoencoders Excel in Cellular Microscopy Representation Learning

New research highlights the superior performance of 3D masked autoencoders (MAE-3D) over their 2D counterparts for learning representations from volumetric microscopy data. Despite cells being inherently three-dimensional, self-supervised learning in fluorescence microscopy often relies on 2D projections. This systematic comparison, using matched architectures and training, shows MAE-3D consistently achieving better results on downstream single-cell tasks. The study further reveals that integrating cross-modal supervision by aligning visual representations with a pretrained protein language model (ESM2) yields even greater benefits for these volumetric models. Key architectural elements like channel cross-attention and frequency-domain regularization were found to be crucial for effectively leveraging 3D spatial context. MAE-3D achieved state-of-the-art performance in protein-protein interaction and protein localization tasks, underscoring the advantages of native 3D modeling and multimodal alignment in single-cell microscopy.