ClusterMAX 3.0: The Industry Standard GPU Cloud Rating System Returns

What it claims

ClusterMAX 3.0 is SemiAnalysis's rating of managed GPU clusters. It rates 77 providers, tracks 323 (up from 209 in 2.0), and draws on interviews with more than 200 end users. Only 19 providers earn a Medallion tier.

Tier movements:

• Google Cloud joins Oracle in Gold.
• Azure drops to Silver.
• Crusoe drops to Bronze.
• Fluidstack moves to Unavailable.
• GMI rises to Silver, joining Lambda, Firmus and TensorWave.
• A new "Participation Ribbon" tier sits between Bronze and Underperforming and holds 15 providers.

The scope is managed clusters only. Inference endpoints, post-training services and sandboxes are out of scope.

How it works

Each provider was asked for:

• 32 Blackwell-class GPUs (B200, B300, GB200 or GB300) or AMD MI355X. Hopper is not accepted.
• An 800G RoCE or XDR InfiniBand network.
• 10TB+ of POSIX/NFS file storage with an RWX CSI StorageClass.
• 10TB+ of S3-compatible object storage.
• A Grafana-style monitoring dashboard.
• 5 days each for testing Slurm and Kubernetes.

Testing has three phases.

Audit. About 15 minutes of pass/warn/fail checks on configuration, versions, scheduler, network, storage, health monitoring and security. This is the only open-source part.

Performance. This covers:

• Grouped GEMMs on cuBLASLt/hipBLASLt and DeepGEMM, using Kimi K2.5/K3 and DeepSeek V3/V4 Pro shapes, plus MAMF, GEMV, nvbandwidth and HPL-MxP.
• NCCL/RCCL tests and RDMA perftest from 8 B to 16 GB, with NVLink disabled on NVL72 to isolate scale-out.
• fio, ior, Elbencho, Torch DCP checkpointing and a Skild AI datagen workload.
• torchtitan training: Llama 3.1 8B with FSDP, and GPT-OSS MoE with expert parallelism.
• InferenceX AgentX for inference.
• Lifecycle timings: pip/uv installs per storage tier, import torch, and vLLM time-to-ready.

Reliability. This phase runs:

• An 8-hour burn-in that stresses GEMMs and all-to-all at the same time. The authors say many providers still burn GPUs and network separately.
• Fabric loops and fio endurance runs.
• Kubernetes pod and PVC churn.
• Full-cluster reboots.
• Failure injection via DCGM, synthetic XID/SXID kernel log lines, or a PCIe bridge reset that produces a real XID 79. Provider tooling must detect and act on its own.

The authors call failure injection their most critical test.

Evidence

The page gives ranges rather than a per-provider table in the visible portion:

• import torch takes 1-2 s on most providers and 10-20 s on filesystems with small-file (LOSF) problems.
• Loading a small model into vLLM from shared storage takes 10-40 s on good providers and over 2 minutes on others.
• Some filesystems lost almost 40% of bandwidth under sustained load. Object storage drifted about 10%.
• GEMM microbenchmarks rarely separate providers. Storage fio sweeps find the most issues.

On Google's GB300, the bundled NCCL picked an unroutable link-local GID and hung. Setting NCCL_IB_GID_INDEX=3 fixed it. Google's gIB plugin then reached 99.5 GB/s on a 16-node, 16 GiB all-to-all, against 95.3 GB/s on other providers.

Oracle's GB300 exposed 4 planes per GPU. Its aggregate PFs had no usable GID, so jobs needed Oracle's SPCX NCCL plugin.

How to use it

The audit is on GitHub and on PyPI:

pip install clustermax

uv pip install clustermax

The performance and reliability suites stay internal. SemiAnalysis also offers standardized Bronze, Silver and Gold SLA templates. These cover node, rack and downtime definitions, acceptance testing and termination rights, available via clustermax@semianalysis.com.

The stated expectations for remediation:

• HGX nodes: failure detection in about 2 minutes, and hot-spare replacement in under an hour end to end.
• NVL72: detection within 2 minutes plus keeping work off the bad node is enough.
• NVL72 contracts: the common pattern is "NVL64+", where the whole rack counts as down once 3 of 18 nodes fail.

On Kubernetes NVL72 clusters, jobs launched without NVIDIA DRA ComputeDomain awareness hung in clique setup when placement crossed racks.

Limits

The fetched text is about 60,000 of 181,968 characters. It stops early in the CoreWeave review, so most per-provider findings and the full tier list are not shown here.

Several factors weigh in the ratings but are not tested hands-on: scale, long-term reliability, support, pricing and delivery timelines. These come from customer interviews. The authors say storage results depend on the volume size each provider gave them.

The CPU (SandboX) and RL (PostTrainingX) suites are announced but not yet part of the ratings. SemiAnalysis also sells the technical assessments it recommends for acceptance testing.