llm-typesafe 0.1a0

How to use it

Install the plugin into the LLM CLI and register a key. Keys come from TypeSafe AI, which is currently waitlisted; the author says the waitlist "seems to move pretty fast".

llm install llm-typesafe
llm keys set typesafe
# Paste key

The model ID is jev. The question goes in the system prompt (-s), and the text being judged goes in as the prompt or is piped on stdin. The answer type is chosen with -o answer_type, and the labels or rubric go in -o criteria as a JSON string. The criteria shape depends on the answer type.

noul (yes/no probability). This example passes no answer_type or criteria option. It returns a probability, not a boolean:

llm -m jev 'Please refund my last payment.' \
  -s 'Does this message explicitly request a refund?'

{"type": "noul", "noul": 0.99}

choice. criteria is a JSON object that maps each label to a natural-language description. Tie-breaking rules go in the system prompt:

cat message.txt | llm -m jev \
  -s 'Which team should handle this message? If billing and technical issues both occur, choose billing.' \
  -o answer_type choice \
  -o criteria '{
    "billing":"Charges, invoices, payments, or refunds",
    "technical":"Problems installing or using the product",
    "other":"Neither category fits"
  }'

score. criteria is a JSON array of rubric levels, ordered from lowest to highest:

cat report.txt | llm -m jev \
  -s 'How reproducible is the problem described in this report?' \
  -o answer_type score \
  -o criteria '[
    "No reproduction instructions",
    "Some instructions, but important steps are missing",
    "Complete steps with expected and actual results"
  ]'

In practice, the label descriptions and rubric wording are the prompt engineering surface. The question itself stays in -s.

Limits

• The version is 0.1a0, an alpha release of the plugin, so expect the option names and output shape to change.
• The page shows output only for noul. It does not show the JSON returned for choice or score, so you can't tell whether you get a single label or index, or per-option probabilities. Check the README (linked but not reproduced) before writing code that parses these.
• The page gives no information on Jev's accuracy, calibration, pricing, rate limits, context length or model size. Nothing on the page backs the noul probability as calibrated.
• You need a TypeSafe AI API key, and getting one depends on the waitlist.