overmind-core/overmind

What it claims

Overmind takes traces from your production agent, scores them against the evals you define, and fine-tunes a model from them. It then serves that model behind one API, and you can download, retrain or roll back the weights. The hosted service (console.overmindlab.ai) and the self-hosted stack run the same code. Self-hosting keeps traces and training data inside your own network.

How it works

Three trace sources feed the platform:

• The SDK instruments the LLM SDKs you already use over OpenTelemetry.
• Any OTel exporter can POST to /api/v1/traces.
• Connectors sync existing traces from Langfuse, LangSmith, Braintrust or Galileo.

The output passed to run.deliver() is the thing that gets scored.

flowchart LR
  A[SDK auto-instrumentation] --> T[Overmind API + Postgres]
  B[Any OTel exporter] --> T
  C[Langfuse/LangSmith/Braintrust/Galileo sync] --> T
  T --> E[Evaluators score run.deliver output]
  E --> F[Modal SFT worker]
  F --> S[Modal vLLM serving]
  S -.retrain / roll back.-> F

The MCP server lives at /api/mcp/. Every tool declares what it costs to run (free, compute, llm or gpu), and no tool can delete anything. Account keys and OAuth connections can reach every project the account is authorised for: call list_projects, then pass project_id. Project API keys stay scoped to their own project.

How to use it

Self-host:

git clone https://github.com/overmind-core/overmind.git && cd overmind
cp .env.example .env    # set the required keys below
docker compose up -d    # Postgres, Redis, API on :8000, Console on :5173, Celery workers, beat, Grafana on :3001

• Swagger is at /api/docs/. Local login accepts any email and password.
• To load a demo workspace: docker compose exec api python manage.py seed_demo --owner <your email>.
• The API refuses to boot until every required key is set, and the error names each missing one.

Fine-tuning and serving need extra setup:

• Deploy overbae/modal/modal_vllm_worker.py, register_model.py and modal_sft_worker.py with modal deploy.
• Create a Modal secret named overmind-inference containing your AWS keys, INFERENCE_API_KEY and HF_TOKEN.

Instrument an agent:

pip install "overmind[tracing]"
export OVERMIND_API_KEY=ovr_…   # add OVERMIND_API_URL for self-host

overmind.init(service_name="support-agent", capability_id="<capability-uuid>", providers="auto")

@overmind.tool()
def search(query: str) -> list[dict]: ...

with overmind.run("support-run", intent=request["question"], conversation_id=session_id) as run:
    answer = agent(request)
    run.deliver(answer)

Without an API key, tracing turns off and the agent still runs normally.

Connect a coding agent over MCP:

• overmind init --ide <cursor|claude|opencode|codex> sets up local config, and overmind sync installs the project key.
• Keys are written only to git-ignored files.
• Claude Code example:

claude mcp add --transport http overmind https://api.overmindlab.ai/api/mcp/ --header "X-Api-Key: ovr_…"

• For a self-hosted instance, replace the host with your API URL (http://localhost:8000 locally).

Limits

• Licence split. Everything outside overmind/ is AGPL-3.0: the platform, including overbae/ and frontend/. Only the SDK, CLI and client libraries under overmind/ are MIT. If AGPL terms don't work for a modified platform you run as a service, Overmind Ltd sells a commercial licence.
• GPU work depends on Modal. Fine-tuning and serving need Modal plus AWS and Hugging Face credentials. A plain docker compose up does not include them.
• Telemetry. The SDK and CLI send anonymous PostHog events (cli.invoked, sdk_init). The page says these never include prompts, trace contents, keys or datasets. Opt out with OVERMIND_ANALYTICS_ENABLED=false or DO_NOT_TRACK=1. Analytics is also off when CI is set.
• OAuth tokens. OAuth connections stay authorised until revoked. Access tokens expire after one hour, and refresh tokens rotate.
• No quality evidence. The page reports no results on how much fine-tuned models improve over the base model, and gives no cost figures for fine-tuning or serving.