VenusRL: A Fully Disaggregated Agentic RL System with Priority Scheduling and Scalable Interaction

What it claims

The speedup range depends on which baseline you compare against:

• Slime: 1.07–3.26× end-to-end training speedup.
• RollFlash: 1.06–4.24×.
• ThunderAgent: up to 2.67×.
• E2B sandbox platform: up to 89% lower environment cost.

So the headline 4.24× is the best case against RollFlash. Against the strongest baseline, the low end is close to parity (1.06–1.07×). The system is 24k lines of Python, Rust and Go.

The motivation comes from the authors' own production profiling:

• On an online SWE-agent workload with a configured generation batch of 64, active batch size averaged only 76.5%.
• KV cache usage stayed near saturation while hit rate collapsed, which the authors call thrashing.
• Sandbox memory on a node averaged about 8% of allocation.
• Sandboxes made up more than 60% of non-GPU cost over three months.

How it works

Training, rollout generation and environment interaction are separate components. A central data coordinator schedules samples.

• Streaming groups to the trainer. Rollout workers compute advantages per finished group and stream it to actor workers, which compute gradients immediately. The update happens once enough groups form a global batch. Weights are then synced back to rollout workers.
• Priority estimation. A lightweight fine-tuned predictor estimates each sample's remaining generation length at turn boundaries. "Critical groups" are the unfinished groups most likely to unblock the next trainer batch.
• Enforcing priority. Priority is applied in three places:
• GPU slots: the RL framework and rollout engine are co-designed so high-priority samples are scheduled first.
• KV residency: a trajectory-aware radix cache and a three-tier KV cache model drive admission and eviction. High-priority trajectories are preserved on a best-effort basis.
• Cross-worker orchestration: the system monitors KV pressure per worker and migrates requests together with their reusable prefix cache.
• Environment side. Sandboxes are microVMs on Kubernetes.
• A global controller allocates sandboxes from a shared pool.
• Per node, placement is page-level, with dynamic admission based on actual memory use rather than declared ceilings.
• Rollouts of the same prompt share snapshot pages: read-only pages map to write-protected physical pages, and copy-on-write creates private copies when a page is modified.

flowchart LR
  DC[Data coordinator] --> RW[Rollout workers]
  PR[Length predictor] --> RW
  RW <-->|tool calls| ENV[MicroVM sandboxes<br/>shared CoW pages]
  RW -->|KV-pressure migration| RW
  RW -->|finished groups + advantages| AW[Actor workers]
  AW -->|batch full: update| AW
  AW -->|weight sync| RW

Evidence

The fetched text gives only the aggregate speedup ranges and the E2B cost figure listed above. It does not state the models, cluster size, task mix, or a per-mechanism ablation. The profiling figures are from a single SWE-agent workload at batch size 32.

Limits

• The fetched page cuts off partway through the design overview. Evaluation details, ablations and any appendix were not available.
• No code release, licence or repository is mentioned in the fetched text.
• Two gains are conditional:
• Priority depends on length-prediction heuristics.
• KV residency for high-priority trajectories is explicitly best-effort.
• Page sharing pays off when rollouts in a group start from the same template. Workloads with heterogeneous sandboxes per group would gain less; this is an inference from the stated design, not something the page tests.
• Adopting it means taking on the whole disaggregated stack: the co-designed rollout engine plus microVM/k8s infrastructure. It is not a drop-in scheduler.
• The authors note that container-level elasticity approaches such as ARL-Tangram don't carry over to microVMs. Conversely, the reader should check whether VenusRL's techniques apply to a container-based setup.