Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)

What it is

A Cloudflare postmortem, published as a coordinated disclosure with a post from SGLang, on cross-request KV cache exposure in the GLM-5.3 Workers AI model (@cf/zai-org/glm-5.3). A researcher, Gadi Evron of Knostic, reported it through HackerOne on 8 September 2026. Cloudflare reproduced it the next morning, took the model offline within 40 minutes of declaring an incident, and restored it on 9 September. No other Workers AI model was affected, and Cloudflare found no evidence of deliberate exploitation.

How it works

Two faults combined. Cloudflare deployed GLM 5.3 with a 1,048,576-token context window, but the prefill KV pool held only 681,280 tokens. SGLang sizes the pool at startup from leftover GPU memory, and nothing forces it to match the context length. Cloudflare knew of the gap, but pre-launch tests checked only HTTP status codes and performance, not output coherence.

The second fault was in SGLang's disaggregated prefill/decode mode:

1. A prompt larger than the pool was marked rejected and swapped for a one-token placeholder.
2. The rejection was stored in to_finish, so req.finished() still returned False and both admission queues treated the request as live.
3. Prefill wrote one KV page and signalled "transfer complete".
4. Decode had reserved pages for the full prompt (8 in the page's example) and attended over all of them.
5. The unwritten pages still held earlier requests' data, because pages are not zeroed between requests.
6. The client got HTTP 200 instead of 400, a coherent reply, and a matching prompt_tokens. The reply came back faster than a full prefill should take.

flowchart TD
    A[Prompt > 681,280 tokens] --> B{Pool holds full context?}
    B -->|Yes, fixed config: ~2.04M| C[Rejection path never reached]
    B -->|No| D[Rejected, placeholder prompt]
    D --> E{Admission checks rejection?}
    E -->|Patched: retired with error| F[Client error]
    E -->|Unpatched: finished is False| G[1 page sent, decode reads stale pages]
    G --> H[HTTP 200 with other users' data]

Preconditions / Affected

• SGLang in disaggregated prefill/decode mode.
• A KV pool smaller than the configured context length.
• A prompt longer than the pool. Here 155 responses to prompts over 681,280 tokens were affected, between 27 August and 9 September 2026.
• Single-node deployments returned the error correctly. Cloudflare's other disaggregated models had pools larger than their context, so they never reached the rejection path.
• The bug was reported upstream as SGLang issue #30233 on 6 July 2026.

What to do

• Upgrade SGLang to v0.5.22 or later. It includes #36988 and #38935, so over-length requests are rejected before handoff regardless of pool size. The patch first shipped in v0.5.20, with follow-ups in v0.5.21 and v0.5.22.
• On every prefill and decode node, compare the KV pool size in the startup log (also exposed as a metric) with the context length. Re-check after any change to weights, quantization, parallelism or memory fraction. If the pool is smaller, lower the context length. Do not rely on a proxy to enforce the limit.
• Run deferred KV release on both roles (default after #41023). Bound Mooncake transfer batches so one timed-out call cannot leave an unbounded tail of writes.
• Enable --disaggregation-enable-kv-checksum and --kv-canary where the model supports them, and check logs to confirm requests are actually verified.
• Configure the load balancer so a client disconnect does not abort a request mid-transfer. Cloudflare holds the prefill connection until prefill finishes and the decode connection until the first token.
• Test long-context output for coherence and correct errors, not only status codes.

Cloudflare's own fix gave GLM 5.3 a prefill pool of about 2.04 million tokens, roughly twice the context. It also added a deployment check covering pool size versus context, deferred release, transfer bounds and required patches.

Caveats

• Checksum and canary checks do not yet cover every architecture. Cloudflare is working to extend them to GLM 5.3, GLM 5.3 Flash and DeepSeek V4.
• #39729 is merged but awaiting release, and zeroing KV pages on release is only a draft proposal (#46435).
• Page-level ownership tags (under 1% throughput cost in Cloudflare's tests) were disabled after reliability issues and are not a general fix yet. The formal verification of the RDMA transfer protocol is a stated goal only.
• Related race conditions across nodes (aborts during RDMA transfer) are a separate bug class, still being fixed upstream.