Retrochainer Post

What it claims

The post argues local LLM hardware is bought for memory, not TOPS: a model either fits or it doesn't, so unified-memory boxes (shared CPU/GPU pool) stretch further than a spec-matched PC with walled-off VRAM. The summary lists the ten rungs; what it doesn't carry is the runnable setup, the model/quant guidance, and the payback math.

How to use it

The setup is identical on every rung. Install Ollama, pull a model sized to memory, repoint your OpenAI client:

curl -fsSL https://ollama.com/install.sh | sh

Model per memory tier:

# 8GB Jetson / 16GB Mac mini:
ollama pull llama3.2
# 24GB 3090 / 7900 XTX:
ollama pull qwen2.5:32b
# 128GB Strix Halo / DGX Spark:
ollama pull llama3.3:70b

The one code change — Ollama serves an OpenAI-compatible endpoint at http://localhost:11434/v1:

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

Optional browser UI:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:main

Model map / quantization: small tier (8–16GB) = Llama 3.2 3B, Gemma 2, Phi-3, Mistral 7B; workhorse (24GB) = Qwen 2.5 32B; heavy (64–128GB) = Llama 3.3 70B, big Qwen/Mixtral. Quantization is the lever: Q4–Q6 is the sane range; below Q4 quality drops faster than memory saved.

Evidence (payback figures)

Against a stated $100–260/month subscription stack (ChatGPT Plus $20 + Claude Pro $20 + Cursor $20 + API $50–200), local runs $2–15/month in electricity. At $100/month the author's payback estimates: $249 Jetson in ten weeks, $599 Mac mini in six months, used 3090 build under a year, $2,000 Strix Halo box ~18 months. Throughput anchor: RX 7900 XTX runs Llama 3.1 8B at ~96 tok/s on ROCm 7.x.

Limits

• The honest asterisk on payback: the box is a sunk cost that only "pays off" if you'd genuinely keep paying the subscription — buying a $2,000 machine to save $20/month is worse than renting. Buy one rung above real needs, not five.
• Prices moving up, not down: the DRAM shortage pushed DGX Spark +18% (to ~$4,699, per Tom's Hardware) and made Apple pull the 512GB Mac Studio, capping it at 96GB.
• AMD saves money, costs time: ROCm 7.x is fine for Ollama/Open WebUI but still trails CUDA on fine-tuning/exotic stacks.
• The 20%: frontier reasoning and cutting-edge coding stay in the cloud; the author recommends keeping one cloud subscription alongside local.
• A GPU (3090/7900 XTX/5090) needs a host PC — add $400–600 before comparing to a finished mini PC.