Switch Models at the Seams

Max Woolf’s post on cheap LLMs (HN discussion) is mostly being read as a story about falling prices, and prices really have fallen. Opus 5.5 is $4/$20 per million tokens with a $0.20 cache read. GPT-6.1 Sol is $2/$10/$0.10. DeepSeek V4.1 Flash is $0.15/$0.60/$0.003. Vercel’s gateway reports cost per token at 43.3% of its April level. The sentence I want to build on is a different one. In coding agents, Woolf writes, “cache read costs are the vast majority (96% – 99%) of the total token expenditure,” with hits “around 90 – 100% of the time.” He gives that as his own observation, not measured data. Even taken loosely, it means a router that treats each turn as a fresh choice of model is pricing the wrong thing.

Here is the mechanism. Once a session is warm, staying on the current model costs a cache read on the context. Switching means paying full input price on the whole context with the new provider, so roughly switch_cost ≈ ctx × (new input price − current cache-read price). At 150k tokens of context on Opus 5.5, staying costs $0.03 a turn. Moving to Sol costs $0.30, and moving to a $10-input frontier model costs $1.50. Moving down to V4.1 Flash costs $0.0225, which is cheaper than staying. Two things follow. The penalty is asymmetric: stepping down is free or better, while stepping sideways or up costs 10–50× a normal turn. And the penalty vanishes at seams, the points where the context gets rewritten anyway: session start, compaction, a subagent spawn, or the periodic context flush that Drew Breunig’s context-engineering survey describes. At a seam the cache is cold whatever you choose, so every model is priced at list.

So the proposal is a router with three rules. Route freely at seams, running the full quality-versus-cost model with no switching penalty. Mid-session, allow downgrades, and make upgrades clear the switch cost: an upgrade has to buy more expected quality than about ten turns’ worth of context reads. Defer escalation to the next seam. If a turn needs a stronger model, compact first. The compacted context is smaller, so the cold read is cheaper, and the seam happens on purpose rather than by accident. The default this implies is start strong and step down, the reverse of the usual cascade that starts cheap and escalates. Cascades were designed for single-shot queries. In a warm session, going down is cheap and climbing up is expensive. Woolf’s data has a market-level hint that this stickiness already shapes behaviour. Twenty-nine providers host V4.1 Flash on OpenRouter, many within 10% of DeepSeek’s price, yet the cheapest endpoints hold “only a minority of the total token share.” Reliability and defaults could explain some of that, but cache locality is a plausible part of it.

Here is how you’d know it worked. Run one task set under three policies (a per-turn router, a static model, and the seam router) and compare realized cost per solved task and cache-hit rate. If the seam router doesn’t beat the per-turn router on cost at an equal solve rate, the asymmetry doesn’t matter in practice. Log every escalation the router deferred and whether the turns spent waiting failed, which directly measures what deferral costs in quality. Count natural seams per session too: if compaction only fires at 200k tokens, the deferral windows are too long and forced compaction becomes mandatory. There is also a lesson for how routers get trained. Preference data is usually collected per turn, as if each model choice were independent of the last. In an agent session it isn’t, so the label should be (model, position in session) and not just the model. One caveat: Woolf lists no cache-write premiums or TTLs, and both would make cold switches more expensive. That changes how big the penalty is. It doesn’t change which way the argument points.