NeuroPB Scales Neural Decoding with Pretrained Behavioral Representations

Brain-computer interfaces (BCIs) face a significant challenge in decoding continuous motor trajectories from neural activity due to the limited and diverse nature of neural recordings. A new framework, NeuroPB, addresses this by transferring knowledge from large-scale, readily available behavioral data. This approach first pretrains a motor encoder on extensive behavioral datasets, which can come from humans, animals, simulations, or robots. Subsequently, NeuroPB aligns neural activity with these pretrained behavioral representations using only a small set of paired neural-behavioral recordings. This allows for the optimization of a neural encoder and a lightweight motor decoder to accurately reconstruct continuous movement. The method has shown substantial improvements in trajectory decoding across multiple macaque motor datasets, including an 11% R^2 increase on center-out tasks. Notably, pretraining on robotic trajectories achieved performance comparable to pretraining on macaque trajectories, highlighting the transferable kinematic structure between biological and artificial systems. The framework also demonstrates improved generalization across different recording sessions, subjects, and motor tasks, requiring minimal calibration. This indicates a promising path towards developing high-performance and calibration-efficient BCIs, especially in scenarios where neural data is scarce.