vLLM Speculative Decoding in 2026: P-EAGLE vs DFlash vs DSpark for Faster, Cheaper LLM Serving

Summary. On July 28, 2026, the vLLM project and its Speculators library shipped open-source support for three parallel drafting algorithms, P-EAGLE, DFlash, and DSpark, that push speculative decoding past a limit EAGLE-3 could not clear. The headline number is concrete: P-EAGLE delivers 1.05x to 1.69x higher throughput than vanilla EAGLE-3 on GPT-OSS 20B, measured on a single NVIDIA B200, and it lifts token acceptance at K=7 by up to 31 percent. Because H100- and B200-class GPUs rent for roughly $2 to $12 per GPU-hour in 2026, a 1.4x throughput gain is close to a one-third cut in serving cost for the same output. The gains are lossless: rejection sampling preserves the verifier model's output distribution exactly, so quality is mathematically identical to standard decoding. This guide explains the three methods, the benchmarks behind them, and the exact vLLM configuration to run them.

Speculative decoding has been a core serving optimization for two years. It works by having a small, fast drafter model propose several candidate tokens, then letting the large verifier model check them all in one forward pass, which turns the memory-bandwidth-bound decode step into something closer to compute-bound. vLLM's original 2024 implementation reported up to 2.8x speedups with methods like EAGLE, Medusa, and n-gram proposals. EAGLE and its successors went further, reaching 2x to 3x over standard autoregressive decoding and shipping in vLLM, SGLang, and TensorRT-LLM. What changed in 2026 is how the draft itself is generated.

The bottleneck: autoregressive drafting hits a wall

EAGLE-3, the state of the art through early 2026, still drafts autoregressively. To propose K candidate tokens, the drafter runs K separate forward passes, one per token. That single design choice creates two production headaches, both named directly by the vLLM team in the P-EAGLE announcement.

First, the drafting cost scales linearly with speculation length. The more tokens you speculate, the more sequential drafter passes you pay for, so the drafter must stay tiny to avoid eating the time that verification saves. Second, choosing the speculation length K becomes a fragile tuning exercise that shifts with server load. Teams end up re-tuning K per workload and per concurrency level, and the autoregressive drafter caps out at a shallow K before the overhead cancels the benefit. In vLLM's own sweeps, vanilla EAGLE-3 reaches peak throughput at K=3, while a parallel drafter keeps climbing to K=7.

The shift: parallel drafting in one forward pass

Parallel drafting removes the sequential loop from the drafting phase. Instead of generating one token per step, the drafter predicts an entire block of K candidate tokens concurrently, in a single forward pass. That decouples proposal latency from the number of tokens speculated, and it changes two things at once: you can now afford a larger, deeper, more accurate drafter because it only runs once per block, and you stop hand-tuning K against fluctuating load. The idea has roots in Medusa and PARD, but P-EAGLE, DFlash, and DSpark combine parallel execution with the deep verifier-state conditioning that made EAGLE effective in the first place.

The three methods were built by teams at Amazon, NVIDIA, and Red Hat AI and released through Speculators, vLLM's training and evaluation library for draft models. They differ in how they feed the verifier's hidden state into the drafter and how they keep training tractable.

P-EAGLE: parallel EAGLE with mask tokens

P-EAGLE keeps EAGLE's core insight, feeding the verifier's hidden states into the drafter, but maps those features across multiple future positions at once instead of one at a time. Its architecture runs in two steps. During prefill, the target model processes the prompt and generates one token, and P-EAGLE captures the internal hidden states along the way, exactly as autoregressive EAGLE does. In the drafter step, it builds inputs for every position in parallel: the next-token position uses the real generated token and its hidden state, while positions 2 through K use two learned placeholders, a shared mask-token embedding and a shared hidden state, so all K tokens emerge from one pass through the transformer layers.

Training a parallel drafter is memory-heavy, because K parallel groups over a sequence of length N create N times K positions. With N of 8,192 and K of 8, one training example holds 65,536 positions, and attention over that is more than four billion elements. P-EAGLE handles this with a sequence partition algorithm that splits a single long sequence into chunks and accumulates gradients across them. In vLLM, integrated since v0.16.0, the drafter is a lightweight 4-layer model trained to predict up to 10 tokens in parallel, and a fused Triton kernel rebuilds the draft batch on-GPU to keep the setup cheap.

The measured results are specific. On GPT-OSS 20B on one NVIDIA B200, P-EAGLE delivers 55 to 69 percent higher throughput at concurrency of 1, with 5 to 25 percent gains sustained at concurrency of 64, against the public vanilla EAGLE-3 checkpoint. The reason is acceptance length, the average number of draft tokens the verifier accepts per round.

Higher acceptance length means more draft work turns into real output. P-EAGLE also benefits more from deeper speculation: from K=3 to K=7 its acceptance length on HumanEval rises by 0.92, while EAGLE-3 gains only 0.38.

DFlash: block diffusion in the KV cache

DFlash routes the verifier's features differently. Rather than feeding hidden states in as ordinary inputs, it projects them and injects them straight into the drafter's KV cache. That conditions the drafter's attention on the verifier's exact state without lengthening the input sequence, and it generates a block of candidate tokens through block diffusion. For training, DFlash uses sequence length sparsification, picking random anchor points along the sequence and computing block predictions only there, which keeps GPU memory in check while still covering the sequence. DFlash landed in Speculators v0.5.0 in May 2026 with both online and offline training, and it is the drafting approach behind serving work such as Laguna XS.2.

DSpark: DFlash plus correction and confidence

DSpark takes the DFlash backbone and adds two pieces. First, a lightweight autoregressive correction head lets later tokens condition more strongly on earlier ones, combining the throughput of parallel generation with the coherence of sequential refinement. Second, a confidence head scores draft tokens before they reach the verifier and forwards only those likely to be accepted, which cuts wasted verification compute under high concurrency. That verification-side saving matters because parallel drafting can cheaply produce many candidates, but the verifier still has to check each one. DSpark is the speculative-decoding method vLLM used for its day-0 Kimi K3 serving in July 2026.

How to serve it in vLLM

Turning this on is a configuration change, not a code rewrite. You download a pre-trained speculator head from Hugging Face and pass a speculative config at launch. For P-EAGLE on GPT-OSS 20B, the vLLM team documents this command.

            vllm serve openai/gpt-oss-20b \
  --speculative-config '{
    "method": "eagle3",
    "model": "amazon/GPT-OSS-20B-P-EAGLE",
    "num_speculative_tokens": 7,
    "parallel_drafting": true
  }'
          

DFlash follows the same shape with its own method flag and checkpoint.

            vllm serve Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 2 \
  --speculative-config '{
    "model": "RedHatAI/Qwen3-30B-A3B-speculator.dflash",
    "num_speculative_tokens": 7,
    "method": "dflash"
  }'
          

Pre-trained heads already exist for common targets, including P-EAGLE checkpoints for GPT-OSS 120B, GPT-OSS 20B, and Qwen3-Coder 30B from Amazon, and P-EAGLE, DFlash, and DSpark speculators for Qwen3-8B, Qwen3-30B-A3B, and gemma-4-31B from Red Hat AI. If you already run a self-hosted stack, the setup slots into the patterns in our production vLLM, Ollama, and LM Studio guide.

Operational notes for production

A few sharp edges are worth knowing before you point real traffic at a parallel drafter. Serving GPT-OSS 20B with EAGLE-style drafters currently needs a one-line vLLM patch (PR#36684) applied before launch, and the fix is expected to land in an upcoming release, so check whether your version already carries it. Parallel drafting also changes the batch shape, because the drafter appends mask placeholders the verifier never checks. vLLM rebuilds the draft batch metadata with a fused Triton kernel and extends the CUDA graph capture range by K times the maximum number of sequences to fit the larger draft batch. Rejected draft tokens are mapped to a padding slot of -1 in the KV cache so they cannot write spurious entries.

The benchmark configuration the vLLM team published is a useful template: an fp8 KV cache, asynchronous scheduling, a maximum model length of 100,000 tokens, and a stream interval of 20, on a single B200. Two levers matter most in tuning. The first is K, the number of speculative tokens: a parallel drafter earns its gains at K=7 where an autoregressive one stalls at K=3, so raising K is often the single highest-value change. The second is the drafter depth, since it now runs once per block, so a 4-layer head that predicts up to 10 tokens can lift acceptance length enough to pay for itself. Measure acceptance length first, then tune K, then decide whether a larger drafter is worth training.

Which method should you reach for

The honest answer is to benchmark on your own traffic, because vLLM's numbers are for GPT-OSS 20B on a B200 and results vary by model, task, and hardware. As a starting point: P-EAGLE is the most drop-in if you already run EAGLE-3, since it reuses the same hidden-state conditioning and has ready checkpoints. DFlash suits teams that want the block-diffusion drafter and are training their own speculators through Speculators. DSpark is the choice under high concurrency, where its confidence head trims verification cost, which is why it backs day-0 serving of a frontier model like Kimi K3. Across all three, the shared benchmark set the team published spans Qwen3-8B on GSM8k math, Qwen3-30B-A3B on HumanEval, and gemma-4-31B on HumanEval, so there are reference points close to most production model families.

The economic case is straightforward. Throughput per GPU is what sets your serving bill, so a 1.4x to 1.69x gain in tokens per second is a direct cut in GPU-hours for the same output, and it stacks with quantization and disaggregation rather than competing with them. For the underlying GPU-cost math, see our breakdown of B200 versus H100 inference cost per token and of FP8 versus BF16 inference cost. The full picture of what runs fastest on which model still starts from the model itself, which is where our Gemini 3.5 Pro, GPT-5.6, and Claude Fable 5 comparison is the pillar reference.

India-specific considerations

For teams serving from India, inference economics are the whole game. Imported GPU capacity is scarce and priced in dollars, so an H100- or B200-hour that costs $2 to $12 abroad carries import and hosting premiums locally, and a 1.4x throughput gain compounds against every rupee of that. Parallel drafting is attractive precisely because it is lossless and requires no change to the served model, so a team running an open-weight model such as Qwen3 or a Kimi variant on rented Indian GPU capacity can cut cost per token without a quality trade-off or a fresh compliance review. Where the served workload touches personal data, keep the model and its logs inside an account or region you control and design the deployment aligned with the Digital Personal Data Protection (DPDP) Act 2023, since a speculator changes speed, not the data path. For the buy-versus-host math on open models, our Kimi K3 self-host versus API cost analysis covers the break-even.

FAQ

How eCorpIT can help

eCorpIT is a Gurugram-based, ISO 27001:2022 certified engineering organisation that deploys and tunes self-hosted large language model serving for teams that need lower latency and cost without giving up control of their data. Our senior engineering teams benchmark speculative decoding on your own workloads, pick between P-EAGLE, DFlash, and DSpark for your models and concurrency, and wire the vLLM serving stack into your infrastructure. Talk to us through our contact page or see our private LLM deployment service.

References

1. Parallel All the Way Down: Beyond Single-Token Generation with Speculative Decoding — vLLM and Red Hat AI, July 28, 2026.

1. P-EAGLE: Faster LLM inference with Parallel Speculative Decoding in vLLM — Amazon and NVIDIA, March 13, 2026.

1. Speculators repository — training and evaluation for vLLM speculators.

1. vLLM repository — inference and serving engine.

1. Speculators model collection on Hugging Face — pre-trained P-EAGLE, DFlash, and DSpark heads.

1. EAGLE-3 speculative decoding on AMD Instinct GPUs — throughput results for Kimi-K2.5 and MiniMax-M2.5.

1. Speculators v0.5.0: DFlash support and online training — vLLM, May 28, 2026.

1. Kimi K3 is here: efficient day-0 support on vLLM — DSpark speculative decoding, July 27, 2026.

1. How speculative decoding boosts vLLM performance by up to 2.8x — vLLM, October 17, 2024.

1. P-EAGLE (arXiv 2602.01469) — parallel speculative decoding paper.

1. DFlash (arXiv 2602.06036) — block diffusion for flash speculative decoding.

1. DSpark (arXiv 2607.05147) — confidence-scheduled semi-autoregressive generation.

1. EAGLE (arXiv 2401.15077) — the base speculative decoding method.

1. Cloud GPU pricing comparison 2026 — H100 and B200 hourly rates.

_Last updated: August 3, 2026._