Building Reasoning Models from the Ground Up: A Video Course

What it claims

The course is eight coding videos by Sebastian Raschka on a reasoning model built from a pre-trained Qwen3 0.6B. One math-answer verifier is built once and then used three ways: to score MATH-500 accuracy, to break ties among sampled answers, and as the reward for GRPO training. Raschka calls it a sequel to his "Coding LLMs from the Ground Up" course, though that course is not required. He frames a reasoning model as the "engine" inside an agent harness, where the harness handles context management and tool access.

How it works

The page lists details by video that go beyond the summary:

• Generation (video 2): greedy decoding, streaming output and end-of-sequence handling. A KV cache is combined with torch.compile, and speeds are compared across hardware.
• Verifier (video 3): extracts the final answer, normalizes it and checks mathematical equivalence. Base and reasoning models are compared on MATH-500. Other evaluation methods are only in the book's appendix.
• Sampling (video 4): temperature scaling, multinomial sampling and top-p filtering produce varied answers. Majority voting (self-consistency) then picks one, and accuracy is weighed against extra inference compute.
• Scoring (video 5): a heuristic scorer and sequence log-probabilities break ties. Self-refinement has the model critique its answer and revise it.
• GRPO (video 6): groups of responses are scored by the verifier and turned into relative advantages. A simplified GRPO loss drives the loop, which logs results and saves checkpoints.
• GRPO additions (video 7): clipped policy ratios, a KL loss term, format rewards that count only when the answer is correct, and tracking of entropy and advantages. Each addition is judged on whether it helps.
• Distillation (video 8): covers soft vs. hard and on-policy vs. off-policy distillation. Teacher answers come from DeepSeek R1 on the MATH dataset. Training is supervised cross-entropy on answer tokens, with train, validation and test splits.

flowchart LR
  V[MATH verifier] --> E[MATH-500 eval]
  V --> S[Self-consistency tie-break]
  V --> R[GRPO reward]
  R --> T[Weight update]
  T -->|next group| R
  D[DeepSeek R1 answers] --> SFT[Distillation SFT]
  SFT --> E
  T --> E

How to use it

The intended workflow is to read a book chapter, code along with its video, then do the exercises. Each video links to its chapter's code in the repository. Python and PyTorch fluency is expected. Dependencies are installed with uv. Video 1 covers checking CUDA or Apple Silicon support and using a remote GPU through VS Code. The early generation and evaluation work runs on a CPU, but a GPU is described as making the later training "much more practical." Raschka also points to a 90-minute conceptual course, "AI Reasoning Models" on LinkedIn Learning, as an optional preview. He says it should be free for 24 hours through his link.

Limits

The page reports no MATH-500 scores, training times or memory figures. Results are in the videos and book only. The GRPO is described as simplified, and video 7 discusses the limits of its simplified KL term and the risk of reward hacking. The material covers math only. The page does not state a code licence.