Speculative Decoding Safety Confirmed at Temperature Zero.

Speculative decoding is a technique used to accelerate large language model (LLM) inference by having a smaller "draft" model propose tokens that a larger "target" model then verifies. A critical safety concern arises regarding whether the draft model's behavior could inadvertently influence the safety-scored outputs, especially when operating at temperature zero (deterministic output). To address this, researchers developed the Typical-Acceptance Invariance Screen (TAIS), a behavioral-equivalence test. TAIS compares outputs from target-only decoding with those from speculative decoding on the same safety benchmarks, requiring byte-identity evidence, statistical equivalence (TOST at +/-3pp), and low Cohen's h values for per-task differences. Applied to a substantial dataset of over 48,000 samples, including various draft types and execution settings, the study found no detectable safety divergence in the tested temperature-zero vLLM stacks. The observed differences in refusal rates were negligible, well below conventional thresholds for meaningful effects. This suggests that speculative decoding, under these specific conditions, maintains the safety integrity of LLM outputs.