PASK Boosts LLM Structured Generation Throughput and Memory

Large Language Models (LLMs) are increasingly used for structured generation tasks, such as producing JSON, SQL queries, or function calls. In these applications, even a single incorrect field can lead to downstream failures. While constrained decoding helps enforce formal validity by tracking parser transitions, the structural information embedded in these transitions, which indicates how tokens contribute to schema-critical decisions, has largely been underutilized in KV (Key-Value) compression strategies. A new technique called PASK (Parser-Aware Structural KV Persistence) addresses this by leveraging parser-derived structural signals to inform layer-group-specific KV persistence decisions. PASK aligns the model's KV sensitivity with the task's structured risk, using task-error sensitivity to set minimum protection floors and attention-output distortion to allocate residual KV capacity. An offline calibration stage compiles these signals into a persistence policy, which then enables lightweight, structure-conditioned lookups during online inference. Evaluations show that PASK, at a target total KV budget of 0.33, outperforms the strongest compressed baseline by 17.39 percentage points on average across various BFCL non-live and Live subcategories using Qwen3-4B models. In end-to-end serving, PASK achieves up to 2.2x higher throughput and 3.3x lower TPOT (Time Per Output Token), while using only 0.53x the peak GPU memory compared to full KV.