JFrog: Why Model Routing Backfires and How to Build Agents That Don't Burn Your Budget

What it claims

Model routing (offloading routine work to cheap models like Claude Haiku, reserving frontier models like Sonnet for hard reasoning) can cut inference costs 40–85% in the right configuration — but implemented naively in a multi-turn agent it can cost more than never routing, because a mid-session model switch throws away accumulated prompt-cache savings and forces the new model to reprocess the whole history at full input-token rates. The summary already has the core mechanism (model-specific prefix caches, Anthropic's ~90% cached-token discount, and the session-level fixes); what it omits is the specific anti-patterns, the named tools, and the concrete numbers below.

The four anti-patterns

The summary names three; the page lists four, and the fourth is the one most likely to surprise:

• Switching models mid-conversation without summarizing — cache-miss penalty on a 20,000-token session can exceed the switch's intended savings.
• Enabling gateway auto-routing mid-session — named tools that do complexity-based routing: LiteLLM, OpenRouter, Portkey. Fine for single-turn queries; in an agentic loop each routing boundary invalidates the KV cache.
• Modifying tools or system prompts mid-session — tool definitions live inside the cached prefix; adding/removing one invalidates everything downstream.
• Compression calls with a separate model — spawning a cheap-model summarization call to compress history generates its own cache miss and resets the main session's cache on return. This is the one the existing summary doesn't mention, and it defeats a common context-management tactic.

Context figure worth noting: agentic coding sessions "can stretch to 50,000 tokens or more," which is where the penalty becomes severe.

JFrog Boost specifics

Boost is pitched as a lightweight CLI that compacts terminal output during agentic loops. The one concrete design claim beyond "reduces bloat": it strips repetitive boilerplate while preserving exact error stack traces — i.e., it trims build/test/log noise before it reaches the model rather than requiring manual curation.

Limits

This is a vendor blog ending in a Boost install pitch. The 40–85% savings range is stated without a source, setup, or workload; the 20,000- and 50,000-token cost examples are illustrative, not benchmarked. No measured before/after numbers for Boost itself are given — the token-reduction claims are qualitative.