New Method Improves Randomized Neural Network Performance and Stability

Randomized neural networks (RNNs) are known for their fast training times and analytical tractability, as they fix input-to-hidden layer parameters randomly and learn output weights directly. However, their performance is highly dependent on a single, uninformed random draw of hidden units, which can lead to redundant representations and inefficient use of model capacity. To address this limitation, a new residual-guided procedure has been introduced. This method iteratively and greedily constructs the hidden layer by selecting units that provide the greatest reduction in the ridge-regularized objective function. At each step, a pool of random candidate units is generated, scored by their exact contribution to reducing the residual error, and the top-k units are selected. The readout layer is then refitted in closed form. This progressive training process guarantees a monotonic decrease in the training objective. The approach is model-agnostic, meaning only the candidate generation is specific to the architecture, while the core selection and refitting loop remains consistent. Extensive experiments on 71 UCI benchmark datasets for both binary and multiclass classification demonstrate that these residual-guided models consistently achieve superior accuracy, stability, and overall ranking compared to their standard randomized counterparts.