Modified RFM Closes Performance Gap with Neural Networks in Noisy Data

Recursive Feature Machines (RFMs) are a type of kernel machine known for their ability to learn features similar to those acquired by Feedforward Neural Networks (FNNs). While RFMs have shown promise in replicating FNN learning dynamics, they often struggle significantly when faced with data-corrupted scenarios, particularly in mathematical problems. This performance disparity has been a notable limitation. To address this, researchers have developed K-Inverse-RFM, a novel modification that involves a simple yet powerful transformation applied to the training labels. This adjustment proves remarkably effective in promoting learning even when data is noisy, complexly represented, or class-imbalanced. The introduction of K-Inverse-RFM allows these machines to not only close the performance gap with FNNs but, in some specific cases, even outperform them, making RFMs a more robust option for challenging data environments.