Skip to content

Back to the catalog

Unsloth AI (@UnslothAI) on X
x.comthread

Unsloth AI (@UnslothAI) on X

Unsloth AI announces a 1-bit dynamic GGUF quantization of Kimi K3, a 2.8-trillion-parameter open model with 1M context and native multimodality, shrinking it from 1.56TB to 594GB (-62%) while retaining ~78.9% accuracy, runnable locally on a Mac Studio with 128GB RAM. Kimi K3 itself introduces Kimi Delta Attention for up to 6.3x faster decoding at million-token context and "Attention Residuals" for ~25% higher training efficiency. The thread links to Unsloth's guide and the Hugging Face GGUF weights.

TL;DR Unsloth ships a 1-bit GGUF quant of the 2.8T-parameter Kimi K3 model, cutting size 62% to run locally on a Mac Studio.
Open original ↗

Organisations named here

  • Unsloth AI
  • Mac Studio

Technologies named here

  • Kimi K3
  • GGUF
  • Hugging Face

Techniques named here

  • Kimi Delta Attention
  • Attention Residuals
  • 1-bit quantization

Related in models