Cache Diagnostics

Anthropic prompt caching only pays off while the beginning of a request stays byte-for-byte identical to a recent one. A reordered tool, a timestamp interpolated into the system prompt, or an edited earlier message silently invalidates the cache — and the only native signal is cache_read_input_tokens dropping to zero, with no indication of what changed.

AgentOS integrates Anthropic's cache-diagnostics beta (cache-diagnosis-2026-04-07) to close that gap. When enabled, each request carries the previous response's message id; the API compares the two requests and returns a cache_miss_reason naming the first point of divergence — the model, the system prompt, the tools, or the message history.

Diagnostics are observability only: they never block, alter, or fail a request, and fingerprints stored by the API contain hashes and token-count estimates, never prompt content.

Cache mechanics themselves — the zero-config defaults, per-call cache options, TTLs, floors, and per-provider behavior — live in Prompt Caching.

Agentic loops: one flag

For generateText (and every agent built on it), set cacheDiagnostics: true. The loop threads each step's provider message id into the next step's comparison automatically — step 1 opts in with previousMessageId: null, step N references step N-1.

const result = await generateText({
  model: 'anthropic:claude-opus-4-8',
  prompt: 'Build the thing.',
  tools,
  maxSteps: 30,
  cacheDiagnostics: true,
  onAfterGeneration: async ({ step, cacheDiagnostics }) => {
    if (cacheDiagnostics?.cacheMissReason) {
      log.warn(
        { step, reason: cacheDiagnostics.cacheMissReason },
        'prompt cache missed — prefix diverged',
      );
    }
  },
});

// The last step's verdict also lands on the result:
result.cacheDiagnostics; // null = prefix stable | { cacheMissReason: {...} }

Per-step verdicts surface on the onAfterGeneration hook (GenerationHookResult.cacheDiagnostics); the final result carries the last step's verdict (GenerateTextResult.cacheDiagnostics). cacheDiagnostics: { previousMessageId } seeds the first step with an earlier response's id, and streamText() takes the same option and resolves result.cacheDiagnostics when the stream ends.

GMI turns

A GMI turn threads the same way. Set cacheDiagnostics in the turn's metadata.options: true opts in with nothing to compare, and { previousMessageId } compares the first step against an earlier response. Each later step of the turn compares against the previous step's response, and each step's verdict and message id arrive on its STEP_FINISHED chunk (cacheDiagnostics, providerMessageId).

Single calls: thread the id yourself

At the provider layer, pass cacheDiagnostics in ModelCompletionOptions and chain ids across calls:

const first = await provider.generateCompletion(model, messages, {
  cacheDiagnostics: { previousMessageId: null }, // opt in, nothing to compare
});

const second = await provider.generateCompletion(model, nextMessages, {
  cacheDiagnostics: { previousMessageId: first.id }, // compare against call 1
});

second.cacheDiagnostics;
// null                                → compared: no divergence
// { cacheMissReason: null }           → comparison still running (check next turn)
// { cacheMissReason: { type, ... } }  → earliest divergence identified

Reading a verdict

cacheMissReason.type is the API's discriminant, passed through as an open string so new types need no library update:

type	Meaning	Fix
model_changed	A router/fallback picked a different model; the cache is per-model.	Hold the model constant within a cached conversation.
system_changed	The system prompt differs — typically an interpolated timestamp or request id.	Keep the system prompt byte-stable; move dynamic data after the cache breakpoint.
tools_changed	Tools added, removed, reordered, or serialized non-deterministically.	Send the same tool list in a fixed order every turn.
messages_changed	An earlier history entry was altered rather than appended to.	Treat history as append-only; echo assistant turns and tool results back verbatim.
previous_message_not_found	No fingerprint for the referenced id (header missing on a prior turn, different workspace, or too much time passed).	Send the flag on every turn; keep turns close together.
unavailable	No comparison produced — another prompt-affecting parameter differs (tool_choice, thinking, beta set, …) or the divergence is beyond the comparison horizon.	Keep prompt-affecting request parameters constant for the conversation.

The *_changed types also carry cacheMissedInputTokens — an estimate of how many input tokens fell after the divergence point. Treat it as a magnitude indicator, not a billing number.

Combine the verdict with usage: diagnostics answer "did my request change?", usage.cacheReadTokens on a generateText result (usage.cacheReadInputTokens on a provider response) answers "did the cache hit?". A null verdict with zero cache reads means the entry expired (TTL) rather than a request bug — consider shorter gaps between turns or a 1-hour cache TTL on the breakpoint.

Notes and limits

• Anthropic API only. Other providers ignore the option; Bedrock/Vertex do not support the beta.
• Send it every turn. The comparison fingerprint is only stored for requests that carried the beta header — skipping a turn breaks the chain with previous_message_not_found.
• Best-effort. A missing verdict (cacheMissReason: null) means the comparison hadn't finished when the response serialized; check the next turn.
• The report names the earliest divergence only. Fix it first — later ones may be hidden behind it.