🧰 TensorRT-LLM

The items that name it 9

• [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
  articleas frameworks and tools2026-10-21It is written here as TensorRT-LLM (TRRTLLM).

  NVIDIA inference library credited by DeepInfer.

• [AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback
  articleas frameworks and tools2026-10-21It is written here as TensorRT-LLM (TRRTLLM).

  Serving stack credited by DeepSeek for DeepInfer.

• DeepInfer: High-performance inference framework for DeepSeek models
  repoas frameworks and tools2026-10-20

  Credited as additional inspiration for DeepInfer.

• [AINews] High Taste Testers Ep. 1
  articleas classification.technologies2026-10-17
• How GLM5.3 Sparse Attention Affects HBM Memory Usage
  articleas classification.technologies2026-09-28
• The Roadmap to Mastering LLM Inference Optimization
  articleas frameworks and tools2026-09-21

  Production inference runtime that uses continuous batching as its default scheduling strategy.

• Vera Rubin NVL72 Agentic Inference: 67x better Performance per Dollar
  articleas frameworks and tools2026-09-14It is written here as TensorRT-LLM (TRTLLM).

  NVIDIA serving stack used for the Rubin and GB300 results; the Rubin build was early pre-release software.

• patchy631/time-to-first-token
  repoas frameworks and tools2026-08-02

  Client-side/embedded inference runtime for optional edge deployment

• Meta Muse Spark Open (816B-A42B) open-weights model card
  repoas frameworks and tools

  Supported inference deployment framework

In every workspace you can read 10