Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)

What it claims

Cloudflare traces the GLM-5.3 (@cf/zai-org/glm-5.3) cross-tenant leak to a rejection path in SGLang's disaggregated prefill/decode mode. In that path, a rejected request still completed the KV handoff. Decode then attended over pages it never received. Any SGLang operator whose prefill KV pool is smaller than the advertised context length could hit the same bug. Cloudflare was also caught out because the failure looked like success. Its pre-launch tests checked only HTTP status codes and throughput, not output coherence.

How it works

SGLang sizes the KV pool at startup from GPU memory left after loading weights. Context length is a separate setting, and nothing forces the two to agree. When a prompt exceeds the pool, Req.set_finish_with_abort records the error in to_finish and swaps the prompt for a one-token placeholder. On a single node, the placeholder drains and the client gets an error. In disaggregated mode, both admission queues check req.finished(). That check reads finished_reason, not to_finish, so the rejected request looks live.

sequenceDiagram
    participant R as Router
    participant P as Prefill
    participant D as Decode
    R->>D: prompt length (N pages)
    D->>D: reserve N pages
    R->>P: request (rejected, 1-token placeholder)
    Note over P: finished() is False, so admitted
    P->>D: 1 page KV + "transfer complete"
    Note over D: finished() is False, so accepted
    D->>D: attend over N pages (N-1 stale)
    D-->>R: HTTP 200, other users' data

KV pages are not zeroed between requests, so the stale pages held earlier prompts and generations. The affected responses had a recognisable fingerprint:

• HTTP 200 instead of 400.
• prompt_tokens matching the long prompt.
• Coherent text.
• Latency faster than the full prefill should take.

Fixes and upstream state

Cloudflare made two changes:

• Configuration: GLM 5.3 now uses the GLM 5.2 configuration. Its prefill pool holds about 2.04 million tokens, against a 1,048,576-token context.
• Engine: A request with a pending rejection is retired at the disaggregation admission queue, before bootstrap, allocation or transfer.

The engine fix has these upstream references:

• #38704: Cloudflare's proposal, which also checks that the transferred token count matches the reserved count.
• #36988: already on main, merged 6 September 2026 for a related failure path.
• #38935: rejects these requests earlier.
• #30233: the same bug, reported by another user on 6 July 2026.

The post describes a related class of page-lifetime and cross-node RDMA race bugs:

• #24226, #30899 and #30937: single-node page-lifetime fixes.
• #31357 and #38961: race fixes, with #38961 superseded by #40645, #41402 and #41404.
• #32564: a reproduction script.
• #32758: bounded Mooncake transfer batches.

Cloudflare also changed its load balancer. A client disconnect no longer aborts a request mid-transfer. Prefill is held until it finishes, and decode is held until the first token.

How to use it

Cloudflare's recommendations for disaggregated SGLang operators:

1. Compare KV pool size (from the startup log) with context length on every prefill and decode node. Redo this after any change to weights, quantization, parallelism or memory fraction. If the pool is smaller, lower the context length.
2. Run v0.5.22 or later, or backport #36988 and #38935.
3. Use deferred KV release on both roles (default after #41023). Bound Mooncake transfer batches inside the release timeout.
4. Enable --disaggregation-enable-kv-checksum and --kv-canary where the model supports them. Confirm in the logs that requests are being verified.
5. Let requests proceed to the first token before the proxy aborts them. This avoids cross-node abort coordination.

Limits

The two corruption checks do not yet cover every architecture. GLM 5.3, GLM 5.3 Flash and DeepSeek V4 are named as still in progress.

Several pieces are unfinished:

• #39729, a checksum cleanup, is merged but unreleased.
• Zeroing pages on release (#46435) is a draft.
• Per-step checksum validation is planned. The #39500 checksum covers only the prefill-to-decode handoff.

Cloudflare tested a page-tag prototype on GLM 5.2 in mid-August 2026. It checks page ownership at read time and cost under 1% in throughput and tail latency. It was disabled for reliability issues and was tied to specific attention backends and architectures.

Formal verification of the RDMA transfer protocol is only planned. The post's version history is inconsistent: it credits v0.5.20, v0.5.21 and v0.5.22 with the fix in different places. Its recommended floor is v0.5.22.