Dicklesworthstone/skillranker

What it claims

SkillRanker (sr) hard-depends on TypeSafe.ai's Jev model — there is no bundled local model, no substitute inference provider, and no shared key. Every user must sign up at the TypeSafe console and supply their own TYPESAFE_API_KEY. Without it, only the offline paths (demo, doctor, roster inspection, replay) work; live ranking is impossible. The summary says Jev is used "for evaluation"; it understates that Jev is a non-optional external dependency the reader must provision and pay for before the tool does anything useful.

How it works

Ranking is a gated two-stage pipeline, not a single scoring pass:

flowchart TD
  A[Context + request + workspace signals + roster] --> B{roster > 254 eligible?}
  B -->|yes| C[Quill BM25 prefilter locally]
  B -->|no| D[full roster to Jev]
  C --> E[wide Choice: candidates + __none__]
  D --> E
  E --> F{needs_skill >= 0.30?}
  F -->|no| G[abstain, no rerank]
  F -->|yes| H[rerank shortlist: up to 8 candidates + fit check]
  H --> I{beats __none__ raw prob?}
  I -->|no| G
  I -->|yes| J[advisory ranking]

Both comparison stages carry a real __none__ option. The needs_skill = mean(specialized_method, material_help, 1 - context_suffices) gate defaults to a 0.30 threshold; below it, sr abstains without spending a rerank call. Surviving candidates must each individually beat __none__'s raw rerank probability. Final rank_score is a softmax over utility_i = ln(p_rerank) + w_fit*log_odds(fits) + ... with defaults w_fit=1.0, priors and phase weighting at 0.0 (opt-in).

How to use it

Install (Linux or macOS; local ranking platform is Linux):

curl -fsSL "https://raw.githubusercontent.com/Dicklesworthstone/skillranker/main/install.sh?$(date +%s)" | bash -s -- --verify

Or build from a pinned toolchain:

git clone https://github.com/Dicklesworthstone/skillranker.git
cd skillranker
cargo install --locked --path . --bin sr

The installer verifies SHA256, installs to ~/.local/bin, and reports hooks as unconfigured — it leaves credentials and network consent to you. Set the key via environment/secret manager, not tracked files (.env is git-ignored; chmod 600 it).

The critical adoption gotcha: installing the hook does not enable live advice. By default the hook runs in shadow mode (evaluates, injects nothing). Live advisory output requires you to explicitly set both, in trusted user config:

[network]
enabled = true
[hook]
mode = "advisory"

A --allow-network flag authorizes only a single CLI run, not the hook.

Limits

• Advisory only: it never executes skills or overrides instructions — as the summary says.
• Claude same-name precedence (project over personal) is not yet backed by conformance evidence, so every rank carries "visibility": "unverified"; confirm a suggested skill actually loads before relying on it.
• Native advice on a harness requires passing real-harness conformance evidence per version; Codex, omp/pi, and Grok integrations each need their own support record. Cass is an archive source only.
• The scoring/gate defaults are described as "workflow choices, not benchmark rankings" — the page presents acceptance thresholds, not measured results.

This covers only the first ~60k of an ~89k-character page; the truncated tail includes further evaluation/promotion-gate machinery (Wilson/Clopper–Pearson bounds, harm monitoring) not fully shown here.