Vera Rubin NVL72 Agentic Inference: 67x better Performance per Dollar

What it claims

SemiAnalysis's AgentX scenario on InferenceX measures Vera Rubin NVL72 serving DeepSeek V4 Pro (1.6T parameters) on pre-release TensorRT-LLM (TRTLLM) with NVFP4. The headline figure is about 67x the throughput per TCO of GB300 at 170 TPS. That figure holds only against GB300 Dynamo TRTLLM, near the end of that engine's measured curve. Against the strongest GB300 configuration in the 60-100 TPS band, where SemiAnalysis says most providers would serve this model, Rubin's advantage is 1.4x to 3x per TCO. SemiAnalysis also measures up to 7x tokens per MW over Blackwell, against NVIDIA's GTC 2026 claim of 3x. The authors call NVIDIA's claim "sandbagging."

Evidence

Per TCO, owning cost. Rubin reaches 276.24 maximum P90 TPS, against 171.53 for GB300 Dynamo TRTLLM. GB300 on SGLang reaches similar interactivity to Rubin. At 80 TPS, Rubin serves 10x more tokens per dollar than single-node B300 and 18x more than H200. At 120 TPS, its lead over H200 is 39x.

Per TCO, 3-year rental. In July 2026, rental cost over $8.5/hr/chip for Rubin and $5/hr/chip for GB300 NVL72. At those rates and 80 TPS, Rubin produces 62% more tokens per rental dollar. At higher interactivity the advantage reaches up to 16x.

Latency. At 60M tokens per TCO dollar, Rubin's P90 end-to-end latency is about 20 s. B200 and B300 on vLLM take 60 s. At 160M tokens per dollar, the gap is 20 s against 120 s.

Per MW at 100 TPS. Values are in million total tok/s per utility MW:

• Rubin: 59.4
• GB300 Dynamo SGLang: 28.5 (Rubin leads by 2.09x)
• GB300 Dynamo TRTLLM: 21.1
• B200 SGLang: 6.95
• B300 vLLM: 5.56
• H200 Dynamo SGLang: 2.26
• MI355X SGLang: 2.01 (Rubin leads by 29.5x)

At 150 TPS, Rubin delivers about 37 per MW, 7.2x GB300 SGLang. The lead narrows to 2.72x at 200 TPS. At 170 TPS, Rubin is 62.9x GB300 TRTLLM per MW but 5.56x GB300 SGLang. The authors state that the engine label is essential when quoting high-interactivity gains. TRTLLM is the stronger GB300 engine at 75 TPS, and SGLang is stronger above that.

Revenue model. The model assumes 75 TPS, 60% utilization and no licence fee, since DeepSeek V4 Pro is MIT-licensed. Per utility GW per year:

• Rubin: $159.5B revenue and $149.9B profit.
• GB300 Dynamo SGLang: $114.9B revenue and $105.3B profit.

That is 39% more revenue and 42% more profit for Rubin. SemiAnalysis computes that Rubin could charge about 28% less per token and match GB300's revenue per GW.

How to use it

The InferenceX calculator lets users replace the default $/chip/hr with their own costs. The two default cost scenarios are hyperscaler-volume owning cost and a 3-year commit rental, both from the SemiAnalysis AI Cloud TCO Model. The tested rack is the production SKU: 2300 W TDP with 1.5 TB of Vera LPDDR5X per compute tray. The article says NVIDIA cut Vera memory in half for this SKU. Rubin also adds "DSX MaxLPS" dynamic power shifting, which provisions GPUs by profiled inference power rather than maximum TDP. SemiAnalysis did not measure it and says a future PowerX integration will.

Limits

• The Rubin software is early pre-release TRTLLM, and only TRTLLM was tested on Rubin. vLLM and SGLang results for Rubin are promised for later.
• NVIDIA staff, including Jensen Huang, Ian Buck and the TRTLLM team, helped bring up the software and verify the results.
• The per-MW table values are interpolated between measured points. "N/A" marks targets outside an engine's measured range.
• H200 runs FP8 and the other configurations run FP4. All comparisons are of whole hardware-and-software configurations, including their caching and parallelism choices.
• DeepSeek V4 Pro has since been replaced by DeepSeek V4.1 Flash. The authors use it as a proxy for models of 1-3T parameters.
• The authors say Rubin's advantage is smaller for offline batch inference.
• The fetched text ends before the fleet-lifecycle revenue section.