Speculative Decoding: PEFT-BD Fails to Deliver Speedup.

Speculative decoding is a technique designed to accelerate the inference of large autoregressive language models by using a smaller, faster "drafter" model to propose multiple future tokens, which are then verified by the larger "target" model. A key goal in this area is to improve draft quality while minimizing additional parameters and system overhead. This paper investigates PEFT-BD, a speculative decoding method that uses a LoRA-like adapter as a block-diffusion drafter within the same backbone as the verifier. Despite several appealing characteristics, such as avoiding tokenizer mismatch, not requiring a separate draft model, and using few trainable parameters, PEFT-BD did not yield a practical speedup in experiments with Qwen3-0.6B. Although it generated non-trivial accepted prefixes, profiling revealed that each speculative step involved a full-backbone draft pass (enabled by the adapter) followed by a full-backbone verification pass (adapter-disabled). This meant the drafter, while parameter-efficient, was not computationally efficient. The study highlights a crucial condition for effective speculative decoding: the drafter's execution cost must be significantly lower than that of the verifier, as longer accepted prefixes alone cannot compensate for a computationally expensive drafter.