Cutting multi-turn Claude agent cost by 71.1% to 74.8% with explicit prompt caching on Vertex AI

The claim

An agent that calls tools sends its whole conversation history back to the model on every turn. On our incident queue, the first request carries 22,497 tokens (5.9k of instructions, 4.3k across 23 tool schemas, and 12.3k of initial bundle context). Each tool turn appends 4,007 tokens on average. Over a 26-turn run, the agent sends 1.59 million input tokens, or 70 times the initial prompt. Prompt caching reuses key-value attention states across requests (Kwon et al., 2023; Gim et al., 2024; Zheng et al., 2024). Unlike Gemini models on Vertex AI, Claude models require explicit cache breakpoints on every request. Placing three cache breakpoints cut our list-price cost per run by 71.1% to 74.8%.

Beyond the unit-cost reduction, this audit established our FinOps verification protocol for tool-using agents. Standard LLM tracing schemas omit provider cache counters by default and price tool-return spans as generation tokens. Reconciling application traces directly against the cloud provider's billing meter isolated three measurement discrepancies and locked in verified unit economics.

What to check this week

1. Read the cloud provider meter beside your traces. The provider meter records cache reads and writes. If it shows almost none, your agent is paying full price for its repeated prefix on every turn.
2. Subtract cache tokens before pricing input. Because pydantic-ai sums cached and uncached input tokens, unadjusted wrappers price cached reads at 110% of base input. Standard Langfuse TOOL spans also price tool-result strings at model rates, inflating apparent Claude spend by 16.2% before caching and 45 to 48% after caching.
3. Name an owner for agent cost. Someone should own the number, its calculation, and its monthly reconciliation against the cloud meter.

What the token meter showed

Claude models on Vertex AI support prompt caching, but unlike Gemini models they do not cache implicitly. Before our change, the Vertex token meter for Opus 4.6 on us-east5 showed 30,460 cache-read tokens against 259 million input tokens over 30 days. That is 0.01% (0.012%).

After we enabled three cache breakpoints in pydantic-ai, 188 evaluation runs read 82.5% and 88.6% of their input tokens from cache. Opus 5.5 on global read 82.5%, and Opus 4.6 on us-east5 read 88.6%. List-price cost per run fell by 71.1% and 74.8%. In 272 follow-on Claude runs two weeks later (report 02), Opus 5.5 and Sonnet 5.5 on global read 79.4% and 79.5% of their input from cache. The first production note inspected after the change read 94.7% of its input from cache across 51 turns.

Early four-turn probes on the Vertex global endpoint showed frequent cache misses, suggesting that global scattered requests across multiple cache pools. At full 16-to-20-turn run length, that effect shrank to a 6.1-point gap in cache-read share and 3.7 points in cost saving.

I lead Applied AI and Support at Yugabyte. The agent is an internal tool that reads diagnostic bundles for our support engineers (report 01). No Yugabyte product is involved in this finding. The mechanism applies to any multi-turn Claude agent running on Vertex AI.

Reconciling trace telemetry against the cloud token meter

Report 01 ended with a stated limitation: our traces carried no cache fields, so billed cost was unmeasured on every arm.

On 2026-09-25, a pricing audit reconstructed 30-day production spend from raw token counts. Across 62 posted notes in the current 3-model shortlist configuration, a note cost $15.68 on average at Vertex list price (median $14.52, 90th percentile $23.70). The Claude Opus 4.6 leg made up 70.6% of that spend ($10.60 per leg across 66 legs). That share held even though two Gemini Flash legs ran alongside Claude on every ticket.

Why did the Claude leg cost so much? The Gemini legs benefited from Vertex automatic implicit caching. OpenTelemetry recorded a 79.9% cache-read share on gemini-3.7-flash and 73.6% on gemini-3.6-flash (plus 82.1% on gemini-3.8-flash vs 61.5% on gemini-4-argon in report 02). Meanwhile, the Claude leg made 26 requests per run (1.59M cumulative input tokens) and paid full input price on every turn.

Checking Google Cloud Monitoring's project-wide token counter for Opus 4.6 on us-east5 confirmed it. Across 29 daily points (259.0M input tokens), 28 days had zero cache reads. All 30,460 cache-read tokens fell on 2026-09-25, the day our first caching smoke test ran.

What the vendors document

Each vendor page below was fetched and verified on 2026-09-29.

Google documents that Claude on Vertex AI "behaves as described in the Anthropic documentation" and that caches are isolated to the Google Cloud project. The default lifetime is 5 minutes (300 seconds), with an optional 1-hour lifetime. A 5-minute write costs 25% more than base input (1.25x), and a 1-hour write costs 100% more (2x). A cache read costs 90% less (0.1x).

Anthropic documents the prompt thresholds. The minimum cacheable prefix is 4,096 tokens for Opus 4.6 and 512 tokens for Opus 5.5. A request below the minimum succeeds without caching and returns no error. A caller can place up to 4 explicit cache_control breakpoints, and prefix lookup checks up to 20 blocks back. On Opus 5.5, a cache read costs 0.05 times base input (95% discount).

On the Vertex pricing page, global list prices per million tokens for Opus 4.6 are $5.00 input and $0.50 cache read ($25.00 output, $6.25 write). For Opus 5.5 on global, prices are $4.00 input and $0.20 cache read ($20.00 output, $5.00 write). Regional us-east5 prices for Opus 4.6 carry a 1.1x multiplier ($5.50 input, $27.50 output, $6.875 write, $0.55 read).

Google adds one compliance note: cache hashes are treated as "User Metadata" (Service Data), so customer-data residency guarantees do not apply to the hashes. Those cryptographic cache hashes contain no reversible customer text, and all prompt payloads pass through our Zero-Data-Retention and two-layer redaction boundary (report 10). Neither vendor page documents how the global endpoint routes cache lookups across regions or pools.

Measurements

Small probes and the scatter hypothesis

Forty-seven probe calls cost $0.65 in total. A prefix of 3,535 tokens on Opus 4.6 cached nothing and returned HTTP 200. A prefix of 5,475 tokens wrote 5,475 tokens on call 1 and read 5,475 on call 2. Opus 5.5 did the same at 1,043 tokens. Repeating an identical request hit the cache in 4 of 4 tries.

Growing four-turn conversations (1.7k to 4.4k tokens, with the breakpoint on the last user message) behaved differently by endpoint. On regional us-east5, Opus 4.6 read the entire prior turn's cache entry on all 3 steps. On the global endpoint, Opus 4.6 missed twice and read a stale, shorter entry once. Opus 5.5 on global missed once, read a stale entry once, and hit once across two identical trials.

Those misses suggested that global load-balanced requests across multiple independent cache pools. A replay model with 3 pools predicted that global scatter would cut Opus 5.5's cost saving to between 34.4% and 48.7% (a 35% to 55% band).

Full-scale evaluation runs

Next, we ran the 188-cell Opus 4.6 vs Opus 5.5 evaluation on 47 closed support cases (2 repetitions per arm, 94 runs each). At full run length, the 3-pool scatter prediction failed.

On global, Opus 5.5 read 82.5% of its input from cache (median 84.8%, minimum 57.3%). That cut list-price cost by 71.1% (median 73.7%, range 42.2% to 82.6%). On us-east5, Opus 4.6 read 88.6% from cache (median 89.9%, minimum 61.8%) and saved 74.8% (median 77.1%, range 44.9% to 81.9%). Zero runs in either arm fell below a 50% cache-read share. Why did the 4-turn probe miss so often while full runs hit 82.5%? Even if a turn lands on a pool that missed the previous turn, on a 16-turn run that pool still holds an earlier turn's prefix.

The 6.1-point read-share gap (3.7 points in saving) between the two rows is confounded by run length as well as endpoint. Opus 5.5 finished in 15.7 requests on average, against 20.2 for Opus 4.6, giving fewer turns to amortize the initial write.

With caching on, cache writes become the largest cost component. Writes make up 55.2% of an Opus 4.6 run (34.2% reads, 10.1% output) and 72.4% of an Opus 5.5 run (13.7% reads, 13.4% output). In addition, our agent enforces a $50 per-run safety cap (UsageLimits). Without caching, 3 of 94 runs hit that cap around turn 34; with caching on, 0 of 94 runs approached the cap.

How the change was wired

In pydantic-ai, AnthropicModelSettings exposes three boolean flags: anthropic_cache_instructions, anthropic_cache_tool_definitions, and anthropic_cache_messages. Enabling all three places 3 of the 4 allowed breakpoints. Those cover the 5.9k system prompt on turn 1, 10.2k system plus tools on turns 2+, and a rolling breakpoint on the last user message.

We kept the 5-minute (300-second) TTL. Across 309 inter-turn gaps in 10 traced production runs, the median gap was 6.4 seconds and the longest was 170.3 seconds (170 s rounded). The cache never expired inside a run, so a 1-hour TTL would have doubled the write premium for zero extra hits.

The first production note inspected after deployment on 2026-09-26 ran 51 requests. It read 4,095,168 tokens from cache, wrote 227,740, and sent 819 uncached (14,444 output tokens). That is a 94.7% read share, costing $4.22 at list price against $24.18 uncached (an 83% saving). Cryptographic cache hashes contain no reversible customer text, and all prompt payloads pass through our enterprise Zero-Data-Retention boundary and two-layer redaction controls (report 10). On 2026-09-27, the production Claude leg switched to Opus 5.5.

Three telemetry reconciliation controls for agent FinOps

Reconciling SDK usage payloads, observability spans, and cloud billing meters established three accounting controls.

1. Disjoint token-bucket accounting across SDK layers (110% vs 10% cache pricing). Anthropic reports input_tokens, cache_read_input_tokens, and cache_creation_input_tokens as disjoint buckets, whereas pydantic-ai sums all three in RunUsage.input_tokens. Passing the summed input_tokens directly into a cost calculator bills cached reads at 100% base input plus 10% cache read (110% total). Our telemetry normalizer subtracts both cache buckets from RunUsage.input_tokens before pricing.
2. Isolating GENERATION spans from TOOL spans in Langfuse. Over the 66 current Claude legs, trace-level Langfuse totals reported $738.50, whereas summing GENERATION spans alone gave $635.74. Langfuse estimates token counts on child TOOL spans and prices those tool-output strings at the model's token rate, inflating pre-cache totals by 16.2%. Once prompt caching cut GENERATION input cost fourfold, that TOOL-span artifact represented 45 to 48% of the unadjusted trace total (report 02).
3. Reconciling pilot extrapolations ($2.5k) against deduplicated production volume ($1.0k to $1.2k). An initial extrapolation from an 8-run pilot replay projected $2.5k a month ($2,532) by multiplying 217 job triggers by an $11.67 per-run saving. In that replay, mean cost dropped from $14.45 uncached to $2.78 cached (80.7% saving, or 81% rounded). Our production audit adjusted for deduplication and sample length. Of 196 committed production executions in the 30-day window (2026-08-27 to 2026-09-25), 141 ran the agent (55 exited on deduplication). Across those runs, 147 Claude legs were billed ($1,393.36 total list-price spend, or $1,393 rounded). The 8 replayed runs also averaged 2.18M input tokens ($14.45), which was 1.36 times the 66-leg mean (1.59M tokens, $10.60). Applying the measured 74.8% evaluation saving to the $1,393 measured 30-day Claude spend gives $1,042. Projecting the current config across all 147 legs ($1,558 base) gives $1,165, and applying the 80.7% replay rate to the $1,481 digest baseline gives $1,196. That bounds the verified list-price saving at $1.0k to $1.2k per month ($1,000 to $1,200), with $1.1k ($1,100) as the central figure.

Verification gates and methodology checks

• The minimum prefix threshold. Turn 1 sends 22,497 tokens, well above 4,096 for Opus 4.6 and 512 for Opus 5.5.
• The 5-minute TTL. The longest gap between turns across 309 transitions was 170.3 seconds, well below 300 seconds.
• The agent framework. The global endpoint misses reproduced in raw AsyncAnthropicVertex SDK calls without pydantic-ai.
• The 3-pool scatter model. It predicted a 34.4% to 48.7% saving for Opus 5.5 on global. Ninety-four full evaluation runs measured 71.1% (82.5% read share), and 272 October runs measured 79.4% and 79.5% read shares.
• The $2.5k/month projection. Replaced by $1.0k to $1.2k/month after removing 55 no-op executions and 1.36x sample-size bias.

Hypotheses falsified by the measurements

• The $2.5k monthly saving headline ($1.0k to $1.2k after reconciliation).
• The 3-pool scatter forecast that global would cap Opus 5.5 savings at 35% to 55% (71.1% measured across 94 runs).
• The 80.7% (81%) replay saving as a headline (5.9 points above the 74.8% measured evaluation rate).
• Trusting trace-level cost totals without separating GENERATION spans from TOOL spans.

Pressure tests

What this does not show

Live post-change production traffic in this bundle is a single 51-turn trace (n=1), which is roughly twice the 26-turn mean length. The 188 evaluation runs (and 272 follow-on runs) are controlled harness runs on closed tickets, not live queue traffic.

The 6.1-point read-share difference between us-east5 and global mixes model version, endpoint routing, and conversation length (20.2 vs 15.7 turns). All dollar figures use public Vertex list prices from 2026-09-29 to provide a contract-neutral baseline independent of private enterprise discount tiers.

Reproduce

Every value printed in this report and its four figures is in numbers.json, recomputed deterministically by gen_numbers.py and rendered by gen_figures.py. To run the same audit on your own agent, sum input_tokens, cache_read_input_tokens, and cache_creation_input_tokens across GENERATION spans only. Price each bucket at your provider's multipliers, and reconcile the monthly total against your cloud provider's token meter.

References

• Gim, I., Chen, G., Lee, S., Sarda, N., Khandelwal, A., & Zhong, L. (2024). Prompt Cache: Modular Attention Reuse for Low-Latency Inference. Proceedings of MLSys 2024. arXiv.
• Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. Proceedings of ACM SOSP 2023. https://doi.org/10.1145/3600006.3613165.
• Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., Barrett, C., & Sheng, Y. (2024). SGLang: Efficient Execution of Structured Language Model Programs. Proceedings of NeurIPS 2024. arXiv.