Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agents

What it claims

Magnitude is a local inference engine that compiles and tunes its kernels on the user's device. The page attributes its lead over llama.cpp to two things. The first is per-device tuning: llama.cpp and similar engines ship kernels precompiled for broad hardware classes. The second is hand-optimized kernels written for the most popular open-weight model families. The FAQ says the second is "how we beat generalist engines." This means the headline speedups (up to 2x, 92% faster decode on Metal, 19% on CUDA) are tied to which model you run, not only to your hardware.

How it works

The page gives only these mechanisms:

• Per-device kernel tuning: kernels are compiled and tuned on your hardware before a model runs.
• Family-specific kernels: hand-written kernels for popular open-weight families. The supported list is at magnitude.dev/models.
• Elastic memory: 27% less memory per agent, and memory is released when an agent stops.
• Shared prefix caches: concurrent sessions share prefix caches, which the page says prevents slowdown when several agents run at once.

How to use it

Setup runs through the desktop app, available for macOS, Windows and Linux:

1. Download, install and open the app.
2. Pick a recommended model in Discover and download it.
3. Connect your agent in Connections.

The magnitude CLI ships inside the app and needs no separate install. The page gives no CLI commands.

One-click connections cover Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi and Cline. Any other client can use the OpenAI-compatible API.

There is no fixed hardware minimum. Any Apple Silicon, NVIDIA or AMD GPU works, or a CPU alone. Available memory sets the model size you can run. No internet connection is needed once a model is downloaded.

Limits

• Benchmark settings are missing. The 92%, 19% and 27% figures come with no model, quantization, chip, context length or batch size. The page links to benchmarks against llama.cpp that are not included here. Check whether your hardware and model match those settings before relying on the numbers.
• Speedups likely depend on model coverage. The advantage is credited partly to hand-written kernels for popular families. Check magnitude.dev/models before assuming a model outside that list gets the same gain.
• Unknowns:
• No figures are given for AMD or CPU-only performance.
• The page does not say how long the on-device tuning step takes.
• It does not say whether a headless or server install exists without the desktop app.