Skip to content

Back to the catalog

datawhalechina/zero-to-sglang
github.comcourse

datawhalechina/zero-to-sglang

Datawhale and RadixArk (the SGLang team's company) run this hands-on course on LLM inference. It starts with KV Cache, prefill vs. decode, and compute-bound vs. memory-bound. It then builds a mini-sglang step by step: forward pass, generation, HTTP serving, Continuous Batching, Paged KV Cache and RadixAttention. The later parts cover the real SGLang source, including quantization, hierarchical caching, DP Attention/EP/PP and Prefill-Decode disaggregation, plus profiling and the PR workflow. It is a rare code-first tutorial on inference engines, available in English and Chinese. Part 0, Part I and Part II chapters 1–2 are done, and the rest is in development.

TL;DR Open bilingual course that builds a mini-sglang from scratch (KV cache, continuous batching, paged KV, RadixAttention), then reads the real SGLang source.
Open original ↗
Document

What it claims

The course teaches LLM inference engine internals by having you write a mini-sglang yourself. You then read the corresponding parts of the real SGLang codebase. It is aimed at people who find inference tutorials either too conceptual or who get dropped straight into the SGLang source. The page argues that a good engine can serve several times more requests on the same GPU. Chapters marked in bold in the syllabus are written by SGLang core members.

How to use it

Prerequisites the page lists:

  • Fluent Python with basic software-engineering habits.
  • PyTorch and neural-network fundamentals.
  • Linear algebra and probability, enough to know roughly how matmul and attention are computed.
  • CUDA is optional. Part I covers the gaps, and attention-backend material only arrives in Part III.

Hardware:

  • Part I needs no GPU.
  • Part II can mostly be debugged on CPU, but the full implementation and performance tests are best run on a GPU. Cloud is fine.
  • Part III is best done on a GPU.

There is no single environment to set up. Dependencies are installed per chapter:

git clone https://github.com/datawhalechina/zero-to-sglang.git
cd zero-to-sglang
# Install dependencies as each chapter requires

Supporting resources:

  • An official reference implementation to check your build against: https://github.com/sgl-project/mini-sglang.
  • The course text is a VitePress site under course-material/, with Chinese in ch/ and English in eng/.
  • community/ collects readers' build logs, pitfalls and study notes.
  • A Discord reader community.

Cohort pacing: the course started on Monday, August 24, 2026, at about one chapter per week.

Contributing

PRs are welcome for errata, wording, translation into course-material/eng/, and community notes. Two constraints apply:

  • The chapter structure is fixed by the writing template. Do not add, remove or merge chapters.
  • Every item in community/eng/PR_requirement.md must be checked before submitting.

Part IV of the course itself walks through profiling, trace analysis and the SGLang PR workflow. Those chapters are still planned.

Limits

  • Licence is CC BY-NC-SA 4.0. That rules out commercial use and requires share-alike for derivatives. Check this before reusing the material in paid training or internal commercial docs.
  • Mostly unfinished. Part II is marked "in progress", and Parts III (advanced techniques) and IV (contributing) are "planned". So the frontier topics and the real-source walkthroughs are not yet available.
  • English coverage is inconsistent on the page. The status line says Part 0, Part I and Part II chapters 1–2 are complete in both languages. The repository tree, however, lists only eng/part0/ and labels the English edition "in progress". Check what actually exists before relying on the English version.
  • The citation entry is a placeholder (author = {TODO}), so the project is clearly still being assembled.

Organisations named here

Technologies named here

Techniques named here

Related in inference