Caching and compression
Pay a tenth for the repeat. Never hit the context wall.
Turn on the response cache and an exact repeat comes back from the cache for a tenth of its charge. Add middle-out and an agent history that outgrows the context window is trimmed from the middle instead of failing.
One repeat on Claude Sonnet 5.5, live
- 6,000 in, 600 out
- $0.009000
- The same request, cache hit
- $0.000900
- Kept by default
- 300 s
- Longest TTL
- 86,400 s
What it does for an agent
- Response cache
- x-uplink-cache: on answers an exact repeat from your organization’s cache, billed a tenth of the original.
- You set the TTL
- 60 seconds to a day per request, or a default on the key. x-uplink-cache-clear forces a fresh answer.
- Headers that explain it
- HIT, MISS or BYPASS, the TTL left, the age of the answer and the request that paid for it.
- Middle-out compression
- transforms: ["middle-out"] drops whole messages from the middle when a prompt is longer than the window.
- Tool calls stay paired
- A tool call and its results are dropped together, so the agent never sees a dangling result.
- Prompt caching too
- Cached input bills at each model’s cached rate, and large shared system prompts can get a cache breakpoint.
Copy, paste, run
OpenAI SDK: cache on
python
1
from openai import OpenAI2
import os3
4
client = OpenAI(5
base_url="https://api.relayswitch.io/v1",6
api_key=os.environ["RELAYSWITCH_API_KEY"],7
)8
9
reply = client.chat.completions.create(10
model="gpt-6.1-sol",11
messages=[{"role": "system", "content": SYSTEM_PROMPT},12
{"role": "user", "content": "Classify: refund request #1182"}],13
extra_headers={"x-uplink-cache": "on", "x-uplink-cache-ttl": "3600"},14
)15
# x-uplink-cache: HIT on an exact repeat within the hour, billed a tenthAnthropic SDK: middle-out
python
1
import anthropic2
import os3
4
client = anthropic.Anthropic(5
base_url="https://api.relayswitch.io/v1",6
api_key=os.environ["RELAYSWITCH_API_KEY"],7
)8
9
reply = client.messages.create(10
model="claude-sonnet-5-5",11
max_tokens=2048,12
messages=long_agent_history, # longer than the context window13
extra_body={"transforms": ["middle-out"]},14
)15
# x-uplink-compressed: how many middle messages were droppedcurl: a miss, then a hit
bash
1
curl -i https://api.relayswitch.io/v1/chat/completions \2
-H "Authorization: Bearer $RELAYSWITCH_API_KEY" \3
-H "x-uplink-cache: on" \4
-H "Content-Type: application/json" \5
-d '{"model": "gpt-6.1-sol", "messages": [{"role": "user", "content": "Capital of Peru?"}]}'6
# First call: x-uplink-cache: MISS x-uplink-cache-ttl: 3007
# Repeat: x-uplink-cache: HIT x-uplink-cache-age: 128
# x-uplink-cache-source-id: the request that paid for itKey API: both on by default
bash
1
# Cache on by default for 10 minutes and middle-out on overflow, for this key.2
curl -X PATCH https://api.relayswitch.io/v1/keys/KEY_ID \3
-H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \4
-d '{"response_cache_ttl": 600, "context_overflow": "middle-out"}'Input, cached input and output, live
USD per million tokens from GET /v1/catalog. A response-cache hit bills a tenth of the original charge instead.
| Model | Input | Cached input | Output |
|---|---|---|---|
| Claude Opus 5.5 | $2.80 | $0.15 | $10.00 |
| Claude Sonnet 5.5 | $1.20 | $0.075 | $3.00 |
| GPT-6.1 Sol | $0.60 | $0.06 | $3.00 |
| Gemini 3.1 Pro Preview | $1.20 | $0.12 | $7.20 |
| DeepSeek V4.1 Flash | $0.06 | $0.0063 | $0.18 |
| Kimi K3 | $0.90 | $0.12 | $4.50 |
| GLM 5.3 | $0.3276 | $0.068 | $1.0296 |
Agent setup guides
FAQs
Questions
For agent builders.
Stop paying twice for the same answer
Cache and compression are opt-in headers or key defaults, on any key.