SGLang
Named by 66 kept items in this workspace.
60 items name it under frameworks and tools and 6 under classification.technologies.
The items that name it 66
Serving; megakernels with aggressive fusion on GB300 NVL72 approach 1,000 tok/s per user on DeepSeek V4.1 Flash.
Serving framework used to measure speculative decoding under production conditions, with CUDA graphs enabled.
Serving framework with Day-0 support for Mistral Large 4; the page says it is the first to serve the model on NVIDIA Vera Rubin.
Inference engine named among fast-growing community and independent projects.
Inference engine with day-one MiniCPM-OCR 1.0 support.
Serving on NVIDIA B300 in BF16 for throughput measurements.
Open-source inference engine used by Cloudflare to serve Workers AI models, including in disaggregated prefill/decode mode. Fixes for this bug shipped in v0.5.20, with follow-ups in v0.5.21 and v0.5.22.
Serving framework; Nanbeige's example splits Nanbeige4.3-33B across 4 GPUs.
LLM serving framework with which ZipFusion is integrated; uncompressed SGLang is the throughput baseline.
Inference engine trusted by the ML community for day 0 open model support.
Serving framework; its TorchTPU plugin reportedly powers production inference at Meta, with a custom MegaKernel.
Serving engine credited by DeepInfer; merged megakernel support, with SemiAnalysis reporting nearly 1,000 tokens/s/user on GB300 NVL72.
Serving engine credited by DeepInfer; merged megakernel support.
Serving project whose team worked with Google on TorchTPU compatibility.
Existing open-source inference tool, compared with DeepInfer.
Credited as additional inspiration for DeepInfer.
Serving engine that merged megakernel support, approaching 1000 tok/s/user on GB300 NVL72.
Serving framework that merged megakernel support, approaching 1000 tok/s/user on GB300 NVL72.
Serves the model with an OpenAI-compatible API (sglang.launch_server), including a Docker image
Serving engine supporting Beam at launch
Inference engine with custom kernels for Muse Spark Open.
Inference engine supported by Muse Spark Open with custom kernels
Serving framework with day-0 support for MOSS-Image from 16 October 2026.
Day-0 serving of Beam with DFlash speculative decoding; also generates rollouts in Miles GRPO
Serving engine used in Rubin preview benchmarks and SGLang Omni for Fish Audio Drama-3.
Serving with text, image, audio and video support; NVFP4 reference configuration on four GB300 GPUs.
High-performance GPU serving framework; version 0.5.22 used for GPU throughput benchmark
Inference backend with day-one support; port 30000
Serving; up to 20% faster FP8 MLA at 128K, 5.9% end-to-end gain from MoE tail fusion
Used to serve the model efficiently
Serving instructions provided for the model
Serving framework under which DRelay's end-to-end speedup was evaluated
Serving framework into which STEPQuant is integrated, using optimized GPU kernels to achieve recurrent-state compression and reduce serving memory
Inference engine used to run DSpark speculative decoding on H100, reaching 2.66x speedup
LLM serving engine whose team designed HiSparse. Used in Dynamo-SGLang results.
inference serving framework used for DSpark speedup benchmarks on H100
Serving framework with DSpark support for LFM2 targets (PR #40651); launched with --speculative-algorithm DSPARK; exposes an OpenAI-compatible endpoint.
serving framework with multi-level hiCache (GPU/RAM/disk) for preserving KV cache across model swaps and restarts
Inference library accompanying open-weight model releases
LLM serving framework, mentioned as supporting the benchmark and as having maintainers who optimize for CUDA on day 0.
Inference/serving system in which Delta Router Replay was implemented to reduce MoE routing export slowdown during agentic RL
Open-source serving stack; GB300 reaches similar interactivity to Vera Rubin on it, and it was the strongest measured MI355X engine at 100 TPS.
Open-source production inference serving engine, with TPU paths through SGLang-JAX and SGL-torchtpu.
Server path mentioned as an alternative to trust_remote_code for Unlimited-OCR.
Used together with Slime to train model weights within Reef.
High-performance, high-throughput inference engine used as LoongSage's inference backend
LLM inference framework/engine whose source the course reads, covering RadixAttention, Paged KV Cache and Continuous Batching
Generation and NCCL weight synchronization engine for online and multi-turn RL, alternative to vLLM for rollouts.
Serving framework that also supports Granite 4.2, with a cookbook providing a ready-to-serve recipe.
Mentioned as an alternative that could be used to serve the embedding model instead of TEI
Serving framework with day-one support for DSpark speculative decoding; used to run GPU inference with target and draft models via an OpenAI-compatible endpoint
Inference serving engine offering an alternative variant of the service; implements RadixAttention (tree-structured prefix reuse) and chunked prefill
Serving framework; used to disable Kimi K2.5's reasoning the same way as vLLM, and referenced as a self-hosting option for GLM-5's chat template.
Inference server (sglang.launch_server, also Docker image lmsysorg/sglang) exposing an OpenAI-compatible API
Serving engine with OpenAI-compatible API; supports YaRN
Recommended production serving engine for Hy4 preview; supports serving via OpenAI-compatible API with speculative decoding (NEXTN) and tool/reasoning parsers
Serving framework to launch a DeepSeek-V4.1-Flash server with an OpenAI-compatible completions API
Serve MiMo-V2.6-Pro-RL with an OpenAI-compatible API via sglang.launch_server; recommended deployment path with a MiMo cookbook for best performance
Serving engine to launch an OpenAI-compatible API server for MiMo-V2.6-Flash-RL, recommended for best performance via the SGLang MiMo cookbook
Inference serving framework used to run Ling-3.0-flash-VL, with a Docker image, launch server command, and a cookbook with hardware- and recipe-specific launch matrix (BF16/FP8 x Low-Latency/High-Throughput)
In every workspace you can read 70
Security
AI
All technologies in this workspace.