Prompt caching
Providers cache the repeated part of your prompt and charge less for it. What ToolsXpo does automatically, and how to read the saving.
Updated September 2, 2026
An agent conversation resends a growing prefix on every turn: the same system prompt, the same tool definitions, the same files. Most providers will cache that prefix and charge substantially less for it.
ToolsXpo handles this for you. There is nothing to configure.
Why it matters most here
Agent traffic is the ideal shape for a prompt cache and the worst shape without one. On real ToolsXpo traffic, agentic requests run around 27% of input served from cache, against 0% for one-shot requests. The difference is not marginal.
Three kinds of model
Caching works differently per vendor, and the gateway translates for you:
- Explicit (Anthropic and family) — nothing is cached unless a breakpoint is marked. ToolsXpo marks them, at the end of the system prompt, the tool definitions, and the last stable turn.
- Automatic (OpenAI, DeepSeek, Gemini, Moonshot and others) — the provider caches a long enough prefix by itself. Markers are ignored at best and rejected at worst, so ToolsXpo sends none.
- None (Groq and some others) — no prompt cache. Markers are stripped.
Your editor behaves identically against all three. That is the point of routing it through a gateway: the client marks a breakpoint, the server decides what that means for the model you picked.
The catch worth knowing
On an explicit provider, writing a cache entry costs a premium and reading it earns a discount. A conversation that establishes a cache and never reuses the prefix costs slightly more than not caching.
This is why caching is a mode per model rather than an always-on optimisation, and why a workspace can turn it off entirely. For normal agent use — where the prefix is reused every turn — it is a clear win.
Seeing the saving
Two places:
- In the editor — the task header shows Cache Reads alongside input and output tokens.
- In your dashboard — AI request logs show the share of input served from cache per request, and in aggregate.
The aggregate figure is weighted by tokens, not averaged per request. A tiny fully-cached call should not outrank a 200k-token one.
What you pay
Cached input is billed at a discount off the normal input rate. The discount is visible in your usage; nothing changes about how you are quoted before a call.
See also
Related
Quickstart: your first task with ToolsXpo Code
Install the extension, sign in from your browser, and give the agent its first task — about five minutes end to end.
Choosing a model
What the model picker shows you, why some models are greyed out, and how to search a catalogue of hundreds with filters.
Credits: two currencies, three sources, and the order they are spent
General credits pay for our compute; AI credits pay for what a provider charges. Where each comes from, when it expires, and which is spent first.
What an agent task costs
How a ToolsXpo Code request is priced, what makes one task cheaper than another, and where to see what you spent.