In 2026 the prompt skill that actually moves API bills is prompt caching: design a stable prefix (tools + system first, dynamic user turns last) and stop breaking it with timestamps, tool shuffles, or mid-chat model swaps. Anthropic and OpenAI both reward exact prefix reuse. Anthropic uses cache_control; OpenAI is largely automatic on supported models. This is an evergreen ops companion to conceptual Context Engineering - distinct from whether prompt engineering is "dead", and not another system-vs-user prompts primer.
In short
- Cache hits need an exact prefix match - order matters
- Anthropic rendered order: tools → system → messages (prompt caching docs)
- Static first; push dates and mode changes into messages
- Common breaks: timestamps, tool shuffle, tool edits, model switch
- Mid-session "cheap model" can cost more via a cold rebuild (mechanism, not a universal dollar law)
- Compaction must share the prefix - or use Anthropic's compaction API
- Claude Code Plan Mode /
defer_loading- label product-specific where needed (Claude Code playbook, Apr 30, 2026)
What is a prompt cache prefix, and why does order matter?
A prompt cache stores the shared front of your request so later turns can reuse it instead of re-billing full uncached input. The match is exact: if you change anything in that shared front, you miss.
On Anthropic (Claude), the docs describe caching the full prefix in this rendered order:
- tools
- system
- messages
On OpenAI (ChatGPT / API), prompt caching is enabled by default for supported models. The entire rendered prefix must still match. Newer families (for example GPT-5.6+) support implicit caching plus explicit breakpoints - same design rule: keep the static front identical.
Think of the prefix like a frozen header on every turn. Anything that must change often - the latest user question, a new file snippet, "today's date" - belongs after that header, not inside it. This is about cache-byte layout, not the role split between system and user text.
How should you order tools, system, and user text?
Stable content first. Conversation last.
| Layer | Put here | Avoid here |
|---|---|---|
| Tools | Fixed schemas you will reuse for the whole session | Random order; add/remove mid-chat |
| System | Role, policies, static project rules | Live timestamps, per-turn mode flags |
| Messages | User turns, fresh file diffs, "it is Monday…" | Nothing that should have been frozen above |
Prefer messages for updates (date, file change, plan mode) instead of editing system or tools. That keeps the expensive shared prefix warm.
Pricing note (multipliers from official caching docs - not invented dollar savings): Anthropic documents roughly 1.25× base input for a 5-minute cache write, 2× for a 1-hour write, and about 0.1× for cache reads (with model footnotes such as Fable/Mythos 5.1 at a lower read multiplier). OpenAI documents similar write ~1.25× / read ~0.1× patterns on supported GPT-5.6+ paths. Absolute $/MTok can change - recheck vendor pricing tables before you model a budget.
What breaks the cache in production?
Anything that mutates the shared prefix. The Claude Code team (April 2026 playbook - evergreen lesson, not breaking news) calls out the classics. Anthropic's invalidation table and OpenAI's optimization notes rhyme.
| Mistake | Why it misses | Safer pattern |
|---|---|---|
| Timestamp or "today" baked into the static system prompt | Prefix changes every day (or every minute) | Put the date in a user/system message for that turn |
| Non-deterministic tool order | Same tools, different bytes → miss | Sort / freeze tool definitions |
| Add, remove, or edit tool schemas mid-session | Tool definition changes wipe Anthropic cache; OpenAI also wants tools stable | Keep the tool list; restrict with tool_choice / allowed_tools (OpenAI) instead of deleting defs |
| Switch models mid-session | Caches are model-specific | Stay on the parent model; hand work to a subagent or a new session |
| Toggle settings that rewrite system+messages (Anthropic examples: some web search/citations toggles, speed settings) | Invalidates beyond the tools block | Read the live invalidation table before flipping flags |
Why can switching to a cheaper model mid-chat cost more?
Because the warm prefix lives on one model. Move the parent session from Opus-class to Haiku-class mid-chat and you rebuild that prefix cold on the new model. For a long agent thread, that rebuild can erase the "I picked the cheap model" savings for that turn.
Claude Code-specific: the team prefers subagents / handoff messages (for example Explore agents on a cheaper model) instead of swapping the parent session model. Treat that as harness design from the Claude Code playbook - not a law of every API app. The transferable lesson is the mechanism: measure cache hit rate before you chase mid-session model hops. Absolute dollars depend on remaining tokens, TTLs, and rate cards - do not claim "always more expensive" as a universal fact.
How do you change plan mode or tools without a cache miss?
General rule: encode mode as messages, not as a system rewrite or a new tool-set.
Claude Code-specific: Plan Mode keeps the full tool set and uses EnterPlanMode / ExitPlanMode tools plus instructions in system or user messages - it does not swap to a read-only tool list. Label that as Claude Code product design when you copy the idea into your own agent.
Large tool sets: Claude Code popularized sending stubs with tool search and defer_loading so the rendered prefix stays stable while full schemas load on demand. That pattern is also documented on Anthropic's tool use with prompt caching and tool search pages. OpenAI documents an analogous defer_loading / tool-search idea. Attribute the origin when you teach it: Claude Code playbook + vendor API docs - not folklore.
| Pattern | Scope |
|---|---|
| Static-first prefix; freeze tools; updates in messages | General (Anthropic + OpenAI) |
| EnterPlanMode / ExitPlanMode without swapping tools | Claude Code-specific |
| Explore / Haiku via handoff instead of parent model swap | Claude Code-specific |
defer_loading + tool search | Claude Code playbook and Anthropic API (OpenAI analog exists) |
| Cache-safe compaction sharing parent system/tools | Claude Code lesson → general design; Anthropic also has a server API |
How do you compact a long chat without paying full uncached input?
The cost trap: a separate "summarize this thread" call with a different system prompt and no tools. That request does not share your warm prefix, so you pay full uncached input for the summary pass - then maybe miss again when you resume.
Better options:
- Anthropic server-side compaction (beta header
compact-2026-01-12) - documented to work with prompt caching. Putcache_controlon system and/or compaction blocks so hits stick. See the compaction docs. - Claude Code-style fork - reuse the same system, tools, and parent prefix; append the compaction prompt as a new user message, then continue. Teach the mechanism even if you are not shipping Claude Code.
Keep a buffer for the compaction prompt plus summary tokens so you do not immediately overflow again.
Anthropic cache_control vs OpenAI automatic caching - what's different?
| Topic | Anthropic | OpenAI |
|---|---|---|
| How you enable | Top-level automatic cache_control or explicit block breakpoints | Enabled by default on supported models; GPT-5.6+ also supports explicit breakpoints |
| Default TTL flavor | 5 minutes (optional 1-hour at higher write multiplier) | Docs cite options such as 30m on newer families - check live docs |
| Design rule | Static prefix first | Static prefix first |
| Tools tip | Changing tool defs wipes cache; defer_loading helps large sets | Keep tools stable; prefer tool_choice: "none" / allowed_tools over removing definitions |
Same PE skill on both sides: freeze the front, mutate the tail. For absolute prices, open each vendor's live pricing or caching table - do not invent cache-read dollar savings in your spreadsheet from this post. Multipliers, TTLs, and model minimums may change - recheck vendor docs before you lock unit economics.
FAQ
Does a cache hit require a 100% identical prefix, or is fuzzy matching enough?
Exact match. Fuzzy or "close enough" prefixes do not count. Anthropic requires identical segments through the breakpoint; OpenAI requires the entire rendered prefix to match. One shuffled tool or one timestamp in the system block is enough to miss.
Should tool definitions or the system prompt come first on Anthropic?
Tools first, then system, then messages - that is Anthropic's documented rendered order for the cached prefix. Put stable tool schemas and static system text early; append conversation last.
Is defer_loading / tool search a Claude Code-only trick?
No - but label the origin. Claude Code popularized the pattern in its April 2026 playbook. Anthropic documents defer_loading and tool search in official API docs, and OpenAI describes an analogous approach. Treat Claude Code as the production story that made the habit famous, not as the only place it exists.
If I only need a quick answer, should I switch the parent chat to Haiku mid-session?
Usually no for a long warm parent session. Caches are model-specific, so a mid-session swap rebuilds the prefix cold. Prefer a subagent, a handoff message, or a fresh short session for the cheap model. Whether dollars go up or down depends on remaining tokens and rate cards - measure; do not assume.
Can Anthropic's compaction API keep my prompt cache warm?
Yes, when you set it up for caching. Anthropic's server compaction beta is documented to work with prompt caching if you place cache_control on system and/or compaction blocks. A one-off summarize call with a different system and no tools is the pattern that throws the warmth away.
How long does a prompt cache stay warm?
It depends on the vendor TTL you chose. Anthropic's default flavor is about 5 minutes, with an optional longer write at a higher write multiplier; OpenAI docs cite options such as 30m on newer families. Treat those as live-doc numbers that may change - confirm on the current caching page before you size idle timeouts.
What to do next
- Freeze tool order and the static system block for each long-lived agent session.
- Move timestamps, mode flags, and fresh file context into messages.
- Measure cache hit rate (Anthropic usage fields like
cache_read_input_tokens/cache_creation_input_tokens; OpenAI's equivalent cached-token fields) before chasing mid-session cheaper models. - Compact with a shared prefix or Anthropic's compaction API - not a cold summarizer.
- When you copy Plan Mode or
defer_loadingideas from Claude Code, label them and re-read the live Anthropic or OpenAI docs for your stack.
Cache multipliers, TTLs, and model minimums can change - confirm on vendor docs before you lock unit economics.
