New Initialization Method Improves Sigmoidal MLP Performance.

Universal approximation theorems confirm the expressive power of sigmoidal multilayer perceptrons (MLPs), but they don't specify how to initialize network weights to reflect the underlying data distribution. This research introduces S-GAI, a spectral geometry-aware initialization framework specifically for one-hidden-layer sigmoidal MLPs. The core idea is to leverage the constructive property of sigmoid units as smooth half-space gates, moving from arbitrary planar geometry to a more informed class-wise spectral geometry derived directly from image data. S-GAI employs Singular Value Decomposition (SVD) to extract essential geometric information—mean, principal directions, and spectral scales—for each data class. An energy threshold then selects the most relevant directions, with each direction represented by two sigmoid gates. These class-specific gates form a shared hidden layer, initialized directly from the training set. Experiments on datasets like MNIST, Fashion-MNIST, and CIFAR-10 demonstrate that S-GAI-initialized MLPs begin with a significantly more informative hidden state compared to Xavier initialization, achieving comparable final accuracy with full training. Even when the hidden layer is frozen, S-GAI still outperforms random gates, confirming its effectiveness in embedding class-wise spectral geometry into the MLP.