Reading SGLang's Speculative Decoding Changes Across v0.5.21 and v0.5.22

Where the speculative decoding changes are listed #

Two SGLang release pages describe a large number of changes, and the speculative decoding entries are spread across several sections. The Release v0.5.21 notes list 779 PRs from 227 contributors, and the Release v0.5.22 notes list 950 PRs from 245 contributors. Both pages place speculative decoding entries under a "Speculative Decoding" heading, and the v0.5.22 page also places related entries under "Breakable CUDA Graph", "Attention Backends" and "MoE & Expert Parallelism".

This article follows one thread through those notes: the path a speculative decoding step takes through a draft stage, a verification stage, an acceptance rule and a state commit. The release notes name these stages through PR titles and highlights. They do not describe the internals of each stage, so this article reports the names, the stated changes and the figures that the notes give, and it does not fill in implementation details.

The mechanism, step by step #

Read as a sequence, the entries in the two releases touch four points in a speculative decoding step:

1. A draft stage produces candidate tokens. The notes name several draft types, including EAGLE, MTP, DFlash and DSpark.
2. A verification stage checks those candidates. The notes refer to target verification, verify kernels and verify CUDA graphs.
3. An acceptance rule decides which candidates are kept. The notes mention opt-in block verification, a greedy one-hot draft mode and a draft sampling change.
4. A commit step records the accepted state. The notes refer to committed length, mamba states and KV cache pools.

The rest of this article takes each step in turn, using only the wording and figures from the two release pages.

Step 1: The draft stage across model families #

The v0.5.21 notes show draft support being added and extended across several model families. The release lists [Spec] Support DFLASH for Kimi K3 (#40794), a LiLiCorr candidate-lattice reranker for DFlash drafts (#37462), and "[LFM2-VL] Add DSpark speculative decoding" (#40651). The same page also lists windowed draft-decode attention for built-in EAGLE and MTP drafts (#32673), and a fix that plans NextN and MTP draft layers as one-layer models (#41194).

The v0.5.22 notes extend this set. The release lists support for EAGLE and MTP in HiSparse, under the title "[HiSparse] Add speculative decoding support with multi-step KV cache swapping" (#37771). It also lists "[Spec] Dspark support" (#41623), and a series of DSpark entries such as "[DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot" (#43670). The series also includes "[DSpark] Fold draft sampling for DeepSeek-V4.1 heads under AUTO" (#42987).

Across both pages, the draft entries cover at least four draft families: EAGLE, MTP, DFlash and DSpark. The notes do not say how these families compare with one another, and this article does not rank them.

Step 2: Verification and the kernels behind it #

Verification entries appear in both releases. In v0.5.21, the page lists "Support XQA backend for SpecDec verify" (#32269), "Use runtime token widths for Triton speculative verification" (#39859), and "Avoid materializing GDN QKV tensors during target verification" (#33778). It also lists "[Fix] Skip the DCP target-verify MLA kernel during FlashInfer autotune" (#41138).

The v0.5.22 notes add more verification work. One entry is "[Spec] Fix wrong tokens from compact-verify CUDA graphs when a token tier is captured more than once" (#42709). Another is "perf(kda): rebalance MTP verify precompute and state synchronization", whose title includes a measured change: the verify kernel went from 13.2 to 9.7 µs on Vera Rubin. The page also lists "[DSA] Fix pooled page table refresh for target verify CUDA graph replay" (#42750).

The verify-kernel figure is a single-kernel measurement named in the PR title. It should not be read as an end-to-end decode figure, because the notes do not give one for the same setup.

Step 3: Acceptance rules and block verification #

The acceptance stage has the most visible new options in v0.5.22. The page lists "[Spec] Add opt-in block verification" (#42297) and "[Spec] Support block verification for the DFlash family" (#43012). The word "opt-in" in the first title means the feature is enabled by a user choice, and the notes do not give a default for it.

A separate DSpark entry adds an environment switch. The title "[DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot" (#43670) names the variable and states that the draft is an argmax draft accepted as one-hot. The notes do not describe the effect on output quality, so this article makes no claim about it.

Acceptance also interacts with the rest of the stack. The v0.5.21 notes list "[Spec] Windowed draft-decode attention for built-in EAGLE / MTP drafts" (#32673) and "[Spec] Fix CDF boundary handling in TreeSpeculativeSamplingTargetOnly" (#35798). Both entries sit between the draft and the accepted output.

Step 4: Committing state #

The commit step is where the notes report the most correctness fixes. In v0.5.22, the highlights include an important fix: hybrid linear models could resume from wrong mamba states when speculative decoding and structured output were used together. The highlight credits @Wirick and Baseten, and the change is listed as "[Scheduler] Use committed length in spec mamba track boundary check" (#43029).

The v0.5.22 page also lists "[Spec][PP] Fix hybrid recurrent-state commit and micro-batch pairing under PP x speculative decoding" (#40001) and "[DFlash] Keep grouped-conv taps inside each request's block so NaN/Inf cannot leak across requests" (#41736). The DFlash entry is one of several that limit the effect of bad values to a single request.

In v0.5.21, the commit area includes "[Fix] Don't write conv state from the fused KDA verify kernel" (#39524) and "Fix mixed chunk prefill with DP speculative coordination" (#41179). The v0.5.21 page also lists "Pipeline parallelism x speculative decoding (EAGLE/MTP) compatibility" (#30775), which covers the interaction between speculative decoding and pipeline parallelism.

Reported figures and their setups #

Both release pages give throughput or latency numbers. Each figure below is tied to the setup stated in the source. The table does not compare figures across rows, because the setups differ.

The v0.5.22 highlight for MiMo-V2.6-Pro and the v0.5.22 FA4 entry both describe 1.85x DFlash decode throughput on 8x B300, which the notes attribute to the change "Use FA4 by default for MiMo on SM100". The notes list the same figure in two places, and this article reports it once.

Integration with parallelism and CUDA graphs #

Speculative decoding entries also touch parallelism and graph capture. The v0.5.21 notes include "[Spec][PP] Launch extend microbatches before the spec output exchange" (#40499), "[Experimental] Preserve speculative decoding during prefill across DP ranks" (#40118), and "[Spec] Model-agnostic last-stage draft embedding under pipeline parallelism" (#39643).

The v0.5.22 notes add CUDA graph entries. "[Spec] Reuse K3 auxiliary outputs across decode CUDA graph sizes" (#40159) and "[Spec] Fix wrong tokens from compact-verify CUDA graphs when a token tier is captured more than once" (#42709) are both listed. The page also lists "[Fix] Skip a draft's decode recapture when it owns no decode graph" (#42347).

These entries show that the draft and verify paths are listed alongside graph capture and replay. The notes do not explain how the two interact in detail, so this article leaves that question open.

Upgrading and running the release #

The v0.5.22 page states a new requirement: to upgrade SGLang, Python 3.11 or newer is now required. It gives the install command below, along with Docker images for NVIDIA CUDA 13, AMD MI35x and MI30x, Intel GPU and Intel CPU.

uv pip install --prerelease=allow sglang==0.5.22
docker pull lmsysorg/sglang:v0.5.22

The v0.5.21 page gives the same install form for version 0.5.21, with its own Docker tags. Teams that need the speculative decoding changes described above should check the relevant PR number in the release notes against their own model and hardware.