Claude mid-conversation tool changes end a 12.5x cache-rewrite penalty in 2026

Summary. On the Claude API a cache read costs 0.1x the base input price and a 5-minute cache write costs 1.25x, so re-processing a prefix you could have read from cache costs 12.5 times more. Editing the tools array is the single edit that guarantees that outcome: Anthropic's own invalidation table says modifying tool definitions invalidates the entire cache, tools, system and messages together. The mid-conversation-tool-changes-2026-07-01 beta, introduced with Claude Opus 5 on 24 July 2026, removes the reason to make that edit. You declare every tool once, then offer or withdraw individual tools with tool_addition and tool_removal blocks inside a role: "system" message. The array never changes, so the prefix hash never changes. On Claude Opus 5 at $5 per million input tokens, a 55,000-token tool prefix costs $0.0275 to read from cache and $0.34 to write again. The beta is available on Claude Fable 5, Claude Mythos 5, Claude Opus 4.8 and Claude Opus 5, across the Claude API, Amazon Bedrock and Google Cloud.

If you run a long agentic session with a changing toolset, this is the highest-value 20-line change you will make this quarter. The rest of this guide covers the prefix model that makes it work, the exact block shapes, the four placement rules that return a 400, the cost arithmetic on current list prices, and how to prove the fix landed using the cache diagnostics beta.

Why the tools array is the most expensive field in your request

Prompt caching hashes the request prefix in a fixed order: tools, then system, then messages. A cache hit requires that prefix to match a recent request byte for byte up to the breakpoint. Because tools sits first, a change there invalidates everything after it.

Anthropic publishes the blast radius directly:

The first row is the one that hurts. A conventional agent that grants a tool when the user enters "deploy mode" does it by appending a definition to tools and re-sending the request. That single append rewrites the hash of the whole prefix, and the API bills you a fresh cache write for the tool definitions, the system prompt and every message in the history.

The docs are also blunt about the second-order cause: tools_changed fires when tools are "added, removed, or reordered between turns, or tool input_schema JSON was serialized non-deterministically." Non-deterministic JSON serialisation is a quiet one. If your service builds schemas from a Python dict without sort_keys=True, two structurally identical requests can hash differently and you pay full write price without ever intending a change.

The cost of one avoidable tool edit

The multipliers are fixed and documented: a 5-minute cache write is 1.25x the base input price, a 1-hour cache write is 2x, and a cache hit is 0.1x. Anthropic's own summary is that caching "pays off after just one cache read for the 5-minute duration (1.25x write), or after two cache reads for the 1-hour duration (2x write)."

Those multipliers resolve to published per-model rates:

Work the arithmetic on a realistic prefix. The tool search documentation gives a concrete reference point: a multiserver setup spanning GitHub, Slack, Sentry, Grafana and Splunk "can consume ~55k tokens in definitions before Claude does any work."

Take a session on Claude Opus 5 where the full cached prefix at the moment of the swap is 120,000 tokens: 55,000 of tool definitions plus a system prompt and a few dozen turns of history.

• Read that prefix from cache: 120,000 / 1,000,000 x $0.50 = $0.06.

• Rewrite it because you edited tools: 120,000 / 1,000,000 x $6.25 = $0.75.

• Delta per avoidable swap: $0.69, or 12.5x the read.

One swap is loose change. A support agent that toggles an escalation tool twice per session across 5,000 sessions a day is 10,000 rewrites, and 10,000 x $0.69 is $6,900 a day in tokens you did not need to buy. Those are list prices worked against a modelled prefix size, not a measurement of your workload, so run the same arithmetic against your own cache_creation_input_tokens before you quote a number internally. The shape of the answer does not change: the write-to-read ratio is 12.5x on every model in the table, because both multipliers scale off the same base input price.

There is a second cost that is easy to miss. Rewriting a large prefix is not just money, it is latency. A cache read skips the prefill work; a cache write does not. Sessions that swap tools every few turns pay that prefill twice per swap, once on the swap turn and once when the next turn re-reads the newly written entry.

How mid-conversation tool changes work

The mechanism is deliberately narrow. You declare the full tool set in tools at the start and never touch it again. To change what the model is actually offered, you append a role: "system" message whose content array carries tool_addition or tool_removal blocks.

            import anthropic

client = anthropic.Anthropic()

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    betas=["mid-conversation-tool-changes-2026-07-01"],
    # The full tool set is declared up front and never changes, so the
    # cached prefix stays intact.
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather for a location.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                },
                "required": ["location"],
            },
        },
    ],
    messages=[
        {
            "role": "user",
            "content": "Say OK.",
        },
        # Withdraw get_weather from this point onward. The block references
        # the tool by name instead of editing `tools`, so earlier turns stay
        # byte-identical and the cache still hits.
        {
            "role": "system",
            "content": [
                {
                    "type": "tool_removal",
                    "tool": {"type": "tool_reference", "name": "get_weather"},
                },
            ],
        },
    ],
)
          

Three details decide whether your integration works on the first try.

Blocks reference, they do not define. Each block's tool field names something already declared. {"type": "tool_reference", "name": "..."} points at an entry in the request's tools array. MCP connector tools can be referenced individually with mcp_tool_reference (server_name plus name) or wholesale with mcp_toolset_reference (server_name). Referencing a name that is not declared in tools returns a 400.

Everything is on by default. Every tool in tools is offered from the first turn unless it carries defer_loading: true, which withholds it until a tool_addition block surfaces it. That flag is how you model a capability that should exist but not yet be reachable: a run_migration tool that only appears after the operator approves a deploy window.

`tool_addition` also undoes a removal. Re-offering a tool that an earlier tool_removal withdrew is the same block. You are toggling visibility on a stable declaration, not editing a list.

Blocks can be mixed with ordinary text blocks in the same system message, so a single append can both explain the change and make it: state "the operator approved a deploy window" as text and attach the tool_addition in the same content array.

The sibling feature: mid-conversation system messages

Tool changes are the beta half of a pair. Mid-conversation system messages are generally available, need no beta header, and solve the same problem one level down the prefix. Instead of editing the top-level system field partway through a session, you append a {"role": "system"} message at the point the instruction becomes relevant.

The priority semantics matter for agent design. Anthropic's guidance is explicit: "a user message is treated as coming from the end user, while a system message is treated as coming from you, the application operator. When the two conflict, system instructions take precedence." A mid-conversation system message keeps operator-level priority without the cache-miss cost of editing the top-level field. Later system messages take precedence over earlier ones, and mid-conversation system messages take precedence over the top-level system field for the turns that follow them.

One availability trap: mid-conversation system messages are not available on Claude Sonnet 5. If Sonnet 5 is your cost tier, this pattern does not exist for you and you use the top-level system field.

Placement: four rules and the 400s they throw

Both features share one placement contract, and it is stricter than most teams expect.

In an agentic loop, the safe slot is directly after the user message that delivers the tool results and before Claude's next turn:

            [
  { "role": "user", "content": "Run the test suite and fix any failures." },
  {
    "role": "assistant",
    "content": [{ "type": "tool_use", "id": "toolu_01", "name": "run_tests", "input": {} }]
  },
  {
    "role": "user",
    "content": [
      { "type": "tool_result", "tool_use_id": "toolu_01", "content": "12 passed, 0 failed" }
    ]
  },
  {
    "role": "system",
    "content": "The user sent the following message while you were working: also update the changelog before you finish."
  }
]
          

That slot is also where you relay input the user typed while the model was mid-loop, which is the second reason to adopt the pattern. Anthropic's phrasing advice is worth copying verbatim into your prompt-engineering notes: state the fact rather than issuing an override, because "Claude is trained to resist instructions that appear to work against the user, and that protection still applies to the system role."

defer_loading, tool search, and choosing between them

Three mechanisms now control what the model can see, and they solve different problems. Picking the wrong one is how teams end up with both a bloated context and a broken cache.

The token case for tool search is strong at scale. Anthropic reports that deferring a five-server catalogue "typically reduces this by over 85 percent, loading only the 3–5 tools Claude needs for a given request", against the ~55k-token baseline quoted earlier. The accuracy case is separate and just as important: "Claude's ability to pick the right tool degrades once you exceed 30–50 available tools."

The published thresholds are usefully concrete. Reach for tool search when you have 10 or more tools, when tool definitions consume more than 10k tokens, or when you aggregate multiple MCP servers, which the docs peg at 200+ tools. Stay with standard tool calling when you have fewer than 10 tools or your definitions total under 100 tokens. The companion context-management page sets the practical line at "roughly 20 tools."

Two limits will bite an aggressive implementation. A request may carry up to 10,000 tools with defer_loading: true, and at least one tool must stay non-deferred or the request returns a 400. Separately, a deferred tool cannot also carry cache_control; that combination is a 400 as well.

The distinction that trips people up: defer_loading "controls what enters the context window, not what you send in the request." You still transmit every definition on every call. What you save is context and attention, not bytes on the wire.

Prove the fix landed with cache diagnostics

Shipping the change is easy. Proving it worked is where teams stop, because the only default signal is usage.cache_read_input_tokens dropping to zero with no explanation.

The cache diagnostics beta closes that gap. Send the cache-diagnosis-2026-04-07 beta header on every turn, pass the previous response id as diagnostics.previous_message_id, and the API reports the first point of divergence.

            r2 = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    system=SYSTEM,
    messages=messages,
    diagnostics={"previous_message_id": r1.id},
    betas=["cache-diagnosis-2026-04-07"],
)
print(r2.diagnostics.cache_miss_reason.type)
          

The reason codes map cleanly onto causes:

Each *_changed reason carries cache_missed_input_tokens, an estimate of how much cacheable prefix was lost. It is derived from byte lengths before tokenization, so treat it as a magnitude indicator rather than a billing number.

Read it alongside usage, because the two answer different questions. Diagnostics answers "did my request change?"; cache_read_input_tokens answers "did the cache hit?" A null diagnostic with low cache reads means your requests matched but the entry expired, which points at TTL rather than at your code. Diagnostics is Claude API only, so Bedrock and Google Cloud deployments still need the manual comparison.

Three caching facts complete the picture. The minimum cacheable prompt is 512 tokens on Claude Opus 5, Claude Fable 5 and Claude Mythos 5, 1,024 tokens on Claude Opus 4.8 and Claude Sonnet 5, and 4,096 tokens on Claude Haiku 4.5; shorter prompts are processed without caching and no error is returned. You can define up to 4 cache breakpoints, and automatic caching consumes one of the four slots. The default lifetime is 5 minutes, measured from the start of the request that writes or reads the entry rather than from the end of its response, and the entry is refreshed at no additional cost each time it is used.

The other 1 July beta: server-side fallback and why it touches your cache

The same release cycle shipped a default mode for the fallbacks parameter, under the server-side-fallback-2026-07-01 beta header. When a safety classifier declines a request, returning HTTP 200 with stop_reason: "refusal", the API retries it on the model Anthropic recommends for that refusal category. The categories are cyber, bio, frontier_llm, reasoning_extraction and general_harms. You can also pass an explicit list of up to three models; the older server-side-fallback-2026-06-01 header accepts only that list form.

This belongs in a cache article for one reason. The cache is per-model, and model_changed is a documented miss reason. A server-side fallback is a deliberate model change inside a single API call, so a fallback that fires mid-session hands the next turn a cold cache on the fallback model. That is the right trade when the alternative is a refused request, but budget for it rather than being surprised by it. Only a safety classifier decline triggers a fallback; a rate limit, overload or server error is returned to you as-is.

Migration checklist

1. Confirm your model is on the list. The beta covers Claude Fable 5, Claude Mythos 5, Claude Opus 4.8 and Claude Opus 5, on the Claude API, Amazon Bedrock and Google Cloud. Claude Sonnet 5 is excluded from the system-message half.

1. Add the mid-conversation-tool-changes-2026-07-01 beta header to the agent's request builder, behind a flag.

1. Freeze the tools array. Declare every tool the session could need, in a fixed order, with deterministically serialised schemas.

1. Mark the tools that should not be reachable at turn one with defer_loading: true.

1. Replace every code path that mutated tools with a role: "system" message carrying tool_addition or tool_removal, appended only in a legal position.

1. Put cache_control on the last block that stays stable across requests, whether that is the end of your tool definitions or the end of the top-level system field. Caching is opt-in; without a cache_control field nothing is cached at all.

1. Turn on cache-diagnosis-2026-04-07 in staging, replay a session that used to swap tools, and confirm tools_changed no longer appears.

1. Watch cache_creation_input_tokens against cache_read_input_tokens in production for a week. The ratio, not the absolute number, tells you whether the change stuck.

Two anti-patterns to leave behind. Do not edit or remove a mid-conversation system message you have already sent; append a new one instead, because rewriting an earlier message invalidates the cache from that point forward. And do not put untrusted text in a system message. Claude treats system content as operator instructions, so raw tool output, retrieved documents and web content belong in tool_result blocks. Teams that skip that rule hand prompt-injection payloads operator-level authority, which is the failure mode we cover in AI agent security and prompt injection guardrails.

What this changes about agent architecture

For two years, the standard answer to "which tools should this agent have?" was "all of them, declared once, because changing them is expensive." That constraint produced agents with 40-tool arrays where the model picks badly, which is exactly the 30–50 tool degradation zone Anthropic warns about.

The constraint is gone. You can now design a session as a sequence of modes with different tool surfaces: a read-only investigation mode, a write mode unlocked after approval, a cleanup mode that withdraws the destructive tools once the work is done. Each transition is two content blocks rather than a full cache rewrite. The real cost of an agent is usually the tokens you re-process, not the tokens you generate.

That framing lines up with what vendors report about the newer models. On the Claude Opus 5 launch, Niko Grupen, head of applied research at Harvey, said the model "matched the output quality of Opus 4.8 running at maximum reasoning while cutting average token usage by 26%." Token efficiency is now a first-class product axis, and prefix stability is the part of it you control in your own code. If you are still choosing between model tiers, our Claude Opus 5 versus GPT-5.6 Sol coding-agent cost comparison works the same arithmetic across vendors, and the broader patterns sit in our enterprise AI agents in production guide.

Teams already tracking Anthropic's parameter churn should read this alongside the Claude API breaking changes for August 2026, because beta headers, not just parameters, are now part of your cache key: the docs list "the set of active anthropic-beta headers" among the prompt-affecting parameters that can produce an unavailable diagnostic. Adding a beta header mid-session is itself a cache event.

India-specific considerations

Two points matter for teams building from India or serving Indian users.

First, data residency has a price. Anthropic's pricing page states that for Claude 4.6 and later models, specifying US-only inference through the inference_geo parameter "incurs a 1.1x multiplier on all token pricing categories, including input tokens, output tokens, cache writes, and cache reads." That multiplier stacks on the caching multipliers, so a US-pinned 1-hour cache write is 2.2x base input rather than 2x. If your architecture pins inference for contractual reasons, the cost of an unnecessary cache rewrite rises by the same 10%.

Second, the system-role warning is a Digital Personal Data Protection Act 2023 issue as much as a security one. Personal data pulled from an Indian user record and pasted into a mid-conversation system message is both a prompt-injection surface and processing you may not have mapped in your consent notice. Keep retrieved records in tool_result blocks, redact before the call, and treat the system role as operator configuration only. The redaction patterns are covered in our PII redaction before LLM calls build guide.

Cache economics also shift the build-versus-buy line for Indian teams running agents at volume. A 12.5x penalty on a 120,000-token prefix is the kind of number that makes an internal platform team worth funding, because the fix is engineering discipline rather than a licence.

FAQ

How eCorpIT can help

eCorpIT builds and hardens production LLM agents for teams in India and abroad, and prefix stability is one of the first things our senior engineering teams audit when an agent's token bill grows faster than its usage. We instrument cache reads against cache writes, remove the code paths that mutate tool arrays mid-session, and design tool surfaces that stay inside the accuracy band the model actually handles. As an ISO 27001:2022 certified, CMMI Level 5 organisation, we design applications aligned with DPDP Act requirements from the first sprint rather than retrofitting them. Tell us what your agent costs today at /contact-us/ and we will tell you what it should cost.

References

1. Anthropic, Mid-conversation system messages and tool changes, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Tool use with prompt caching, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Prompt caching, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Pricing, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Cache diagnostics (beta), Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Tool search tool, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Manage tool context, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Refusals and fallback, Claude Platform Docs, retrieved 6 August 2026.

1. Anthropic, Claude Platform release notes: API, retrieved 6 August 2026.

1. Quartz, Anthropic launches Claude Opus 5 at half the price of Fable 5, 24 July 2026.

_Last updated: 6 August 2026._