Cryptanalytic Method Extracts GLU Feed-Forward Blocks from LLMs.

Previous cryptanalytic extraction methods have successfully recovered components from ReLU networks, networks with componentwise activations like GELU or SiLU, and the final projection matrices of Transformers. However, these methods could not extract the bias-free Gated Linear Unit (GLU) feed-forward blocks commonly used in many modern language models. GLU blocks feature a unique two-branch structure, multiplying an activated linear projection by a second learned linear projection within each hidden unit, which posed a challenge for existing techniques. This research presents a novel, constructive, multi-stage forward-query recovery primitive specifically designed for isolated bias-free GLU blocks. The method uses finite-difference curvature to identify gate-direction candidates. Paired observations at `x` and `-x` then enable the separation of gate magnitude, orientation, and value-branch coupling. Experiments demonstrated high accuracy, with sub-percent median validation error across high-precision targets, Qwen layers, a Llama subproblem, and a full-dimensional Gemma block. While four finite-precision configurations remained below 5 percent median error, none perfectly reproduced every stored weight. It's important to note that these experiments focus on isolated blocks and do not represent an end-to-end model-API attack, as deriving the necessary internal block responses from final model outputs remains an unsolved problem.