Response cache

When an agent sends a chat completion it already sent, byte for byte the same request under the same policy, Intertrace can answer with the reply it checked the first time instead of calling the model provider again. The answer arrives in milliseconds and costs nothing at the provider.

The request is still decided exactly as without a cache: the gateway authenticates the key, checks the agent's policy, spend limits and preset, and runs the input checks (data protection, prompt attacks, your organization's rules). Only when all of them pass the request with nothing masked or flagged does Intertrace look in the cache. The stored reply is then checked again for this request (data protection, the output checks and the system-prompt leak check); if it isn't a clean pass, the request goes to the provider as usual.

1. 1

   Open Costs

   Open Costs → Budgets and find Response cache (it is off by default).
2. 2

   Switch it on

   Choose how long answers are kept: from 1 minute to 24 hours, 1 hour unless you change it.
3. 3

   Decide on varied answers

   Optional: also cache requests whose temperature is above 0. Their answers normally vary, so they are left out unless you include them.
4. 4

   Save

   The change is in your change log with an Undo, and the gateway applies it within seconds.

Someone who manages costs can change the setting. The card shows how many requests the cache answered and what that saved.

• The same organization and the same agent: an answer is never shared across organizations or agents.
• The same request body, compared after canonical JSON ordering (RFC 8785), so key order and spacing don't matter but any other change does.
• The same model, the same provider route and the same key paying for it.
• The same agent policy: changing anything the gateway decides with (rules, data settings, the preset, this setting) starts a fresh cache.

• Streamed replies (stream: true) and requests asking for more than one choice.
• Anything but an OpenAI-compatible chat completion: the Responses API, Anthropic messages, embeddings and others always go to the provider.
• Replies that call a tool, and conversations that carry tool calls or tool results.
• Requests or replies where anything was masked, flagged or blocked, including personal data or credentials.
• Replies that didn't finish normally (cut off at the token limit, refusals, errors), or larger than 256 KB.
• Requests with a temperature above 0, unless you include them.
• Requests paid by the free platform trial.

Every reply that could use the cache carries x-intertrace-cache: hit or x-intertrace-cache: miss. miss means the reply didn't come from the cache, and it also appears while your organization's cache is off; hit appears only while it is on.

A request answered from the cache is recorded once in Traffic, as allowed and answered from the cache. No provider was called, so it has no cost and never counts toward spend or a budget. The provider cost it saved, at list price, is noted on the record and shown on Costs, apart from spend.

Intertrace checks this every day: a request answered from the cache that was billed anyway is a reporting error and is flagged on the status page's Reporting accuracy component.