x.comthreadJASON MCNAB (@JASONMCNAB) on X
Tweet reporting VIDRAFT's POCKET-Darwin-180B, a 4-bit GGUF (111GB) of a 180B MoE built on Qwen3.8-Flash-Next. Self-reported figures: 4.17 tok/s on a gaming laptop with an 8GB GPU and 32GB RAM using llama.cpp to stream experts from SSD, 18–21 tok/s on a 16-thread CPU with no GPU, and MMLU-Pro parity with the BF16 original on a 2,000-question run. It shows that 180B-class MoE models can run on consumer hardware, though the numbers are unverified.
TL;DR VIDRAFT's POCKET-Darwin-180B 4-bit GGUF (111GB) reportedly runs at ~4 tok/s on an 8GB-GPU laptop via llama.cpp expert streaming from SSD, and 18–21 tok/s on CPU.
Organisations named here
- VIDRAFT
Technologies named here
- MMLU-Pro
- llama.cpp
- POCKET-Darwin-180B
- Qwen3.8-Flash-Next
- GGUF
- POCKET-Darwin-180B BF16 original
Techniques named here
- 4-bit quantisation
- Expert streaming from SSD
Also named here
- Mixture of Experts
Related in models
