x.comthreadSemiAnalysis (@SemiAnalysis_) on X
SemiAnalysis claims AMD chips running vLLM can earn up to $3 per chip-hour in profit serving the open-source Kimi K3 model, even after higher chip rental prices following Claude Code's rise. Kimi K3 has over 2.8 trillion parameters and fits AMD's 288GB per chip, while B200 cannot hold it in one server. B200 must then use tensor or pipeline parallelism over 400Gbit/s InfiniBand or Ethernet, which SemiAnalysis says is 18x slower than NVLink. The post gives no methodology.
TL;DR SemiAnalysis argues AMD GPUs running vLLM can profit up to $3/hr/chip serving the 2.8T-parameter Kimi K3 because of 288GB memory per chip.
Organisations named here
- SemiAnalysis
- AMD
- NVIDIA
- vLLM project
- Anthropic
- Claude Code
- vLLM
Technologies named here
- Kimi K3
- vLLM
- Claude Code
- InfiniBand
- NVLink
Techniques named here
- Tensor parallelism and pipeline parallelism
Related in inference
arxiv.orgpaperMinimally Invasive Steering of Language Models
x.comthreadSamuel Zeng (@SamuelZengML) on X
huggingface.coarticle