After enforcement is installed and the unit economics are decomposed, every strategist arrives at the same question: where do I actually find 50% cost reduction without degrading quality? In 2026, the answer is two levers — model tiering and prompt caching — and they are not nearly as widely deployed as they should be. Model tiering means routing easy queries to cheap models (GPT-4o-mini, Claude Haiku 4, Gemini 2.5 Flash) and reserving expensive frontier models (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro) for the queries that actually need them. Caching means using Anthropic's prompt caching with cache_control or OpenAI's prompt cache to avoid re-paying for the same input tokens — system prompts, retrieved documents, conversation history — when they repeat across calls. Either lever alone moves a typical agent's bill 25-40%. Both together, executed well, move it 50-75%. This lesson is the strategist's playbook for both: when to tier, how to route, what to cache, what the cache hit rate should be, and the failure modes that quietly destroy the savings. The math is concrete because the savings are concrete.

Why These Two Levers Are the Strategist's Go-To

Of the five components in the cost-per-run decomposition, model tokens are the line the strategist can move fastest without organizational change. Reducing human review requires a policy conversation. Reducing overhead requires consolidation. Reducing tool latency requires upstream system changes. Reducing model tokens requires only two things: a routing decision and a cache configuration. Both can be implemented by a small team in days, not quarters.

The reason tiering and caching are underused in 2026 is not technical difficulty. It is organizational inertia. The team that built the agent picked a model — usually the smartest one available because the demo went well on hard cases. They never revisited the choice because the agent works. Tiering "feels risky" because it explicitly routes some queries to a smaller model. Caching "feels complicated" because cache_control requires a small amount of careful prompt structure. Both fears are overblown by a factor of 5x relative to the actual savings.

The strategist who introduces tiering and caching to a portfolio of 15 production agents in the first 90 days will typically deliver 35-55% reduction in monthly LLM spend with no measurable quality regression. That is the kind of number that earns the next budget cycle's seat at the table.

Lever One: Model Tiering — Route by Difficulty

Model tiering is the simplest cost lever and the one most teams skip. The premise: not every query needs the frontier model. A "what's the status of my order" query and a "explain this discrepancy in our reconciliation" query do not require the same compute. Routing the first to Haiku 4 at $0.80/$4 per million tokens and the second to Sonnet 4.5 at $3/$15 captures 3.75x cost reduction on the easy queries while preserving quality on the hard ones.

The three-tier model that works in 2026

• Tier 1 (frontier reasoning): Claude Sonnet 4.5, GPT-5, Gemini 2.5 Pro. Use for genuinely hard reasoning, novel problems, multi-step planning, anything where being wrong matters more than being slow. Typically 15-30% of queries on a tiered agent.
• Tier 2 (cheap-fast workhorse): Claude Haiku 4, GPT-4o-mini, Gemini 2.5 Flash. Use for routine classification, simple Q&A, formatting tasks, drafting where humans will review. Typically 50-70% of queries.
• Tier 3 (specialized cheap): Embedding models, classifier models, single-purpose fine-tunes. Use for pre-processing, intent classification, retrieval scoring. Typically 10-20% of queries.

Worked math. A customer service agent processing 200K queries/month, currently 100% on Sonnet 4.5 at $0.022/query = $4,400/month. Move to tiered model: 60% to Haiku 4 at $0.005/query, 25% to Sonnet 4.5 at $0.022, 15% to a fine-tuned classifier at $0.0008/query. New blended cost: $0.0094/query = $1,880/month. Savings: $2,520/month, 57% reduction. Same workload, same quality on the queries that matter.

How to do the routing

Three routing patterns, ranked by sophistication.

1. Static routing by intent. A pre-step classifier (cheap, fast) determines intent. Each intent has a pre-assigned tier. "Order status" → Haiku 4. "Refund dispute" → Sonnet 4.5. Simple, deterministic, easy to audit. Works for agents with bounded intent vocabularies.
2. Dynamic routing by complexity score. A cheap router model (often GPT-4o-mini itself) scores the query for complexity on a 1-5 scale. Queries scoring 1-2 go to Tier 2, 3-4 to Tier 1, 5 to Tier 1 with explicit reasoning enabled. Works for open-ended agents. Adds 5-15ms latency and $0.0002 per query — easily recovered.
3. Cascade with fallback. Try Tier 2 first. If the output fails an eval check (confidence threshold, hallucination detector, format validator), retry on Tier 1. Works for agents where you can verify output quality cheaply. Adds 1.2-1.5x cost on the retry but skips 70-85% of Tier 1 calls.

The right pattern depends on the agent. Most teams should start with static-by-intent because it is the easiest to audit and explain. Move to dynamic or cascade once the agent has stable intent-level metrics.

The Tiering Failure Modes

Tiering goes wrong in three predictable ways. Knowing them in advance prevents them.

Failure 1: Quality regression on the wrong subpopulation

The router sends a query to Tier 2 that Tier 2 cannot handle. The output is wrong in a way the eval suite misses. The customer experiences degraded quality. Mitigation: pair tiering with a stratified eval that explicitly samples Tier 2 outputs at 2x the normal rate and compares them to Tier 1 outputs on a held-out set. Watch for quality drift.

Failure 2: Tier inflation

The team adds complexity ("just send this category to Tier 1 to be safe") until every query is back on Tier 1. The savings evaporate over 6 months. Mitigation: publish tier distribution monthly. Set a target (e.g., "60% on Tier 2") and treat regression on that target as a tracked metric.

Failure 3: Latency degradation from the router

The pre-step router adds 200-400ms latency that an unattended agent doesn't care about but a human-facing agent does. Customer-facing chat agents notice the lag. Mitigation: use streaming classifier models (Cerebras-hosted Llama or Groq-hosted Mixtral) that return classifications in under 50ms, or use cheap synchronous embeddings + similarity routing instead of generative classification.

Lever Two: Prompt Caching — Anthropic and OpenAI Mechanics

Prompt caching is the lever that confuses teams because the mechanics differ between vendors and the savings are nonlinear. The basic idea: if you send the same input tokens repeatedly (a long system prompt, a stable knowledge base chunk, conversation history) the vendor can cache them server-side and charge you a fraction of the input price for the cached portion. The savings depend on what fraction of your input is cacheable and how often the cache hits.

Anthropic prompt caching with cache_control

Anthropic's implementation, available on Sonnet 4.5 and Haiku 4 in 2026, uses an explicit cache_control breakpoint in the prompt structure. You mark sections of your prompt (typically the system prompt, retrieved documents, and tool definitions) with cache_control: ephemeral, and Anthropic caches them for a default 5-minute window. Reads from the cache are 0.1x the normal input price. Writes (the first call that warms the cache) are 1.25x the normal input price.

• Pricing impact: Sonnet 4.5 input is $3/M base, $0.30/M cached read, $3.75/M cached write.
• Break-even: Cache pays off after 2 reads. If your same system prompt + docs are reused 5x within 5 minutes, you save ~70% on that portion.
• Long cache option: 1-hour cache available at 2x write cost, 0.1x read cost. Pays off after 4 reads in the hour.

OpenAI prompt cache

OpenAI's implementation is automatic on GPT-4o, GPT-4o-mini, GPT-5, and o-series models. No cache_control directives needed. The cache key is the first N tokens of the prompt; if they match a recent call (within ~5-10 minutes), the cached portion is charged at 0.5x the input price (less aggressive discount than Anthropic but no setup).

• Pricing impact: GPT-5 input $5/M base, $2.50/M cached. GPT-4o-mini $0.15/M base, $0.075/M cached.
• Break-even: Immediate. Cache hit is always cheaper than miss.
• Prompt structure matters: Put stable content first (system prompt, then retrieved context, then variable user input last). The longest stable prefix is what gets cached.

What to cache

The high-leverage cacheable content categories, in order of typical savings.

1. System prompts. Agents have long system prompts (2,000-15,000 tokens of instructions, examples, persona). They are sent on every call. Caching them is the biggest single lever.
2. Tool definitions. Function calling agents send tool schemas on every call (often 1,000-4,000 tokens). Cacheable.
3. Retrieved context (when stable). If your retrieved documents are the same across multiple calls in a session (multi-turn agent over the same knowledge), cache them. Don't cache one-off retrievals.
4. Few-shot examples. 5-10 examples included in the prompt for in-context learning. Cacheable.
5. Conversation history. The growing transcript of a multi-turn conversation. Cache the stable prefix; only the new turn is uncached.

What not to cache

• Variable user input (it changes every call by definition).
• One-off retrievals where the same doc won't appear in subsequent calls.
• Anything sensitive that shouldn't be retained beyond the immediate call.

Cache Hit Rate: The Number That Matters

The metric the strategist tracks is cache hit rate — the percentage of input tokens served from cache. A well-tuned agent on Anthropic typically achieves 70-90% cache hit rate. A naive deployment achieves 0-20%.

Worked example. A customer service agent on Sonnet 4.5, averaging 4,200 input tokens/call (90% stable: system prompt + tools + retrieved KB; 10% variable: user query + conversation tail). At 200K calls/month:

• Without caching: 200K * 4,200 tokens * $3/M = $2,520/month input cost.
• With 85% cache hit rate: 200K * 4,200 * 0.85 cached at $0.30/M + 200K * 4,200 * 0.15 uncached at $3/M = $214 cached + $378 uncached = $592/month input cost. Plus amortized write cost (1 write per ~50 reads at 5-minute TTL) negligible.
• Savings: $1,928/month, 76% reduction on input cost.

Output cost is unaffected by caching — it's only the input side that benefits. For input-heavy agents (most agents), this is still the dominant lever. For output-heavy agents (code gen, long-form), caching matters less.

Combining Tiering and Caching

The two levers compound. Tiering reduces the per-call cost; caching reduces the input portion of every call. Apply both.

Worked: the same 200K-query customer service agent.

• Baseline (Sonnet 4.5 only, no caching): $4,400/month total LLM spend.
• Tiering only (60% Haiku, 25% Sonnet, 15% classifier): $1,880/month.
• Caching only (Sonnet 4.5 with 85% cache hit rate on input): $1,872/month (input drops from $2,520 to $592, output stays $1,800).
• Tiering + caching together: $885/month. ~80% reduction from baseline.

The combined approach is not additive — it is multiplicative on the savings on each tier. The Haiku 4 calls also cache their stable inputs at proportionally lower cached-read pricing. Both vendors offer caching across their full price tiers.

The Caching Failure Modes

Caching goes wrong in four predictable ways.

Failure 1: Cache invalidation from prompt drift

Every minor change to the system prompt invalidates the cache. Teams that A/B test prompt changes weekly see cache hit rates that bounce around. Mitigation: version system prompts deliberately, batch changes, accept a cache-warming penalty after each prompt update. Don't make system prompt changes lightly.

Failure 2: Cache TTL expiration on bursty workloads

5-minute TTL means a workload with 7-minute gaps between similar queries gets zero cache benefit. Mitigation: use the 1-hour cache option (Anthropic) for stable workloads or batch queries to land within the TTL window for batch jobs.

Failure 3: Variable position destroying the cache

A common mistake: putting the user's variable input early in the prompt followed by the stable retrieved context. The cache key is the prefix; if the prefix changes every call, nothing caches. Mitigation: prompt structure discipline — system → tools → retrieved → user input, in that order, always.

Failure 4: Per-tenant cache pollution

Multi-tenant agents where each tenant has slightly different context get poor cache hit rates because the cache is fragmented. Mitigation: structure tenant-specific context as a small variable suffix, keep the bulk stable across tenants where possible.

The 90-Day Rollout Plan

For the strategist taking over a portfolio of agents with no tiering and no caching, here is the realistic 90-day plan.

Days 1-15: Measurement and baseline

• Pull current model spend per agent from the gateway.
• Decompose input vs output token spend per agent.
• Identify the top 5 agents by spend. Focus on those first.
• Establish cache hit rate baseline (probably 0% if no one has tuned it).

Days 16-45: Caching rollout

• For each top-5 agent: restructure prompts (stable prefix, variable suffix), add cache_control breakpoints (Anthropic) or rely on automatic caching (OpenAI).
• Target cache hit rate: 60% by day 30, 80% by day 45.
• Verify no quality regression — caching is server-side and should not change outputs, but verify your eval suite is unchanged.

Days 46-75: Tiering rollout

• For each top-5 agent: classify queries by intent or complexity. Identify the 50-70% that can route to Tier 2.
• Implement static-by-intent routing first; move to dynamic complexity routing in cycle 2.
• Run a 2-week shadow comparison: dual-call Tier 1 and Tier 2, compare outputs, ensure no quality drift on the routed population.
• Cut over to tiered routing with circuit breaker (revert to all-Tier-1 if quality drops).

Days 76-90: Measure and publish

• Calculate savings per agent.
• Publish before/after to FinOps council.
• Plan rollout to remaining agents in next quarter.

Typical outcome: 40-60% LLM spend reduction across the top-5 agents, $30K-$200K/month savings depending on baseline. The next quarter the strategist applies the same playbook to the next 10 agents. By month 6 the portfolio is fully tiered and cached.

When Not to Tier or Cache

Two cases where these levers should not be the first move.

Low-volume agents. An internal research agent at 4K calls/month has $200/month LLM spend. Spending 5 engineering days to save 50% gets you $100/month back. The engineering time costs more than the savings recover. Tier and cache only when the math favors it — typically $1K/month spend or higher per agent.

Agents with strict latency budgets and small input. Voice agents with sub-300ms latency budgets and 800-token prompts get poor cache hit rates (short prompts) and can't afford the router latency. For these, focus on model selection (already on Haiku/Flash) and skip the caching dance.

For everything else — moderate-to-high volume, input-heavy, latency-tolerant — both levers should be defaults, not optimizations.

Key Takeaways

• Model tiering and prompt caching are the two cost levers that move 50%+ of an agent's LLM bill without organizational change. They are underused in 2026 due to inertia, not difficulty.
• Three-tier model in 2026: Tier 1 frontier reasoning (Sonnet 4.5, GPT-5, Gemini 2.5 Pro), Tier 2 cheap workhorse (Haiku 4, GPT-4o-mini, Gemini Flash), Tier 3 specialized (embeddings, classifiers, fine-tunes). Typical distribution: 15-30% Tier 1, 50-70% Tier 2, 10-20% Tier 3.
• Three routing patterns: static by intent (simplest, audit-friendly), dynamic by complexity score (open-ended agents, 5-15ms overhead), cascade with fallback (verify cheaply, retry on failure). Most teams should start static.
• Anthropic prompt caching uses explicit cache_control breakpoints, 0.1x read price, 1.25x write price. Break-even at 2 reads. Sonnet 4.5 input $3/M base → $0.30/M cached.
• OpenAI prompt cache is automatic on GPT-4o family and GPT-5, 0.5x cached price, prefix-keyed. Less aggressive discount but zero setup.
• Cache hit rate is the metric to track. Naive deployment 0-20%, well-tuned 70-90%. Prompt structure (stable prefix, variable suffix) and prompt-change discipline determine the rate.
• What to cache, ordered by leverage: system prompts, tool definitions, stable retrieved context, few-shot examples, conversation history prefix. What not to cache: variable user input, one-off retrievals, sensitive data.
• Combined tiering + caching is multiplicative, not additive. The same agent on Sonnet 4.5 dropping from $4,400/month to $885/month is the typical outcome — about 80% reduction.
• Four caching failure modes: prompt drift invalidating cache, TTL expiration on bursty workloads, variable position destroying cache, per-tenant cache pollution. All preventable with structure discipline.
• 90-day rollout: 15 days measurement, 30 days caching rollout (target 80% hit rate by day 45), 30 days tiering rollout with shadow comparison, 15 days measure and publish. Typical outcome: 40-60% spend reduction on top-5 agents.
• Skip these levers for low-volume agents (under $1K/month spend) and ultra-low-latency voice agents with small prompts. Everything else: tier and cache as defaults.