llama.cpp
Named by 60 kept items in this workspace.
51 items name it under frameworks and tools and 9 under classification.technologies.
The items that name it 60
Local inference runtime with merged support for Mistral Large 4 (PR #31376).
Local inference; PR #31376 adding Mistral Large 4 support was merged.
Local GGUF inference; LFM3 support merged via PRs #31257, #31252 and #31275.
Local inference engine with day-0 support for Mistral Large 4 GGUF quants, with a built-in WebUI, image input and tool calls.
Inference engine named among fast-growing community and independent projects.
Local inference engine; supports MiniCPM-OCR 1.0 and merged LFM3 support.
On-device inference engine for LFM3-VL-2B.
On-device inference, measured at Q4_0 with 32K context.
Supported backend for Ollama, as listed in the Ollama README.
Let the author run language models locally; prompted them to buy a gaming machine with an NVIDIA card to build with AI.
Supported runtime for granite-speech-5.0-2b.
Local inference; Unsloth shipped an implementation for DeepSeek V4.1 Flash with dynamic GGUFs.
Unsloth shipped an implementation for DeepSeek V4.1 Flash 1017 with dynamic GGUFs.
Backend for running open-weight models locally through AnyLanguageModel.
Local inference runtime that Muse Glimmer works with
Inference engine run on the console's GPU through its Vulkan backend (ggml-vulkan); supports every architecture and quantization it knows.
Day-0 GGUF support for Muse Spark Open
Inference engine with prebuilt binaries and GGUF quants (Q2_K, Q4_K_S, Q8); llama serve gives an OpenAI-compatible server and built-in WebUI.
GGUF checkpoints for efficient edge inference
Inference backend with day-one support; serves on port 8080
Local inference; merged Decision 3.0 support via PRs #30378 and #30383
Runs the GGUF model locally, streaming experts from the SSD on a laptop with a small GPU.
Provides llama-server, a self-run local server Apogee can connect to for larger models
Local LLM inference engine; featured on Microsoft's Windows ML stage and received a PR adding a GPU-side cache for MoE experts kept in host memory
Open-source framework that runs large language models locally on your computer; used here to serve Qwen 3.8-27B over localhost.
Provides day-one support to run both Open d1 models anywhere
Inference engine with day-0 support for EmbeddingGemma 2 and a merged PR enabling it
Used to serve the model efficiently
Local LLM inference engine; added a /v1/systemone endpoint for local decision-model inference and MTP speculative decoding support
Local inference engine; PR #27773 adds GLM-5.3-Flash/GLM5-Next support enabling local inference for the 320B hybrid text+vision model.
Inference engine used as the comparison baseline; Magnitude claims up to 2x faster decode than it.
Runs GGUF-quantized models for local inference, named as a deployment target for GGUF-format models.
Inference engine used for GGUF model quantizations, including CLM and DSpark deployments
local inference engine for quantized GGUF models
Inference engine with day-one DSpark support (requires build from PR#29339); run with llama-server using --spec-type draft-dspark.
runs Hy4 Preview via custom CUDA kernels for the mixed-precision quantization scheme
Inference engine that powers local AI tools; provides a foundation for local inference with its own runtime, memory management, and broad hardware support; used as the performance reference (via llama-bench) that transformers is compared against.
target platform for planned inference-only C/C++ bindings
Local inference runtime for GGUF-format models
Local inference engine used to serve Jev-compatible decision models such as Rizzo Flow and jevos.
Local inference engine used for running quantized Qwen models; noted for fragile cache reuse at high context
Local inference with GGUF builds, including vision mmproj weights
Serves GGUF models locally and exposes an OpenAI-compatible server (llama-server) used both for the IFStruct evaluator and for serving the base and fine-tuned models
Native inference reference; its speculative-decoding graph for Qwen 3.5/3.8 was the reference for Pooled's
Framework used to convert Granite 4.2 models to the GGUF format for reduced-memory deployment, in several quantization levels (Q8_0 through Q2_K).
Inference engine with day-one support for LFM-compatible DSpark draft models; used on-device with experimental Metal kernels for speculative decoding
Runs the GGUF model locally (llama-server and llama-cli).
Referenced as an alternative framework; TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp
Tool for hardware-aware installation and GGUF model selection through verified local inference, OpenAI-compatible serving, reproducible tuning, and multi-GPU operation
Supported inference deployment framework
Local inference engine used to run the GGUF model on CUDA, Metal, or CPU, including a server with OpenAI-compatible API and web UI, and a CLI
In every workspace you can read 75
AI
Security
All technologies in this workspace.