synapse / insights

LLM Cost Insights

Illustrative cost breakdowns of the failure patterns we see most often in agent setups — the ones that blow up token budgets, and the infrastructure levers that stop the bleeding.

15x

modelled cost variance

Between identical agent runs with different model mixes

$1,200+

weekly ghost spend

From silent retry loops and confused sub-agents

57%

typical token reduction

From proxy-level pruning and caching

The figures on this page are illustrative worked examples of common failure patterns — modelled scenarios, not measured results from real customer accounts. Run your own prompt through the live sandbox to see numbers from your own traffic.

worked examples

Where the money really goes

worked example

Multi-agent SaaS: 15x cost variance between runs

A B2B SaaS team shipped an autonomous research agent that chains planning, search, and drafting sub-agents. Two identical user requests could cost $0.80 or $12.40 depending on which models the router picked and how many search loops the planner triggered.

before

$31,000 / month

after

$8,200 / month

the fix

Synapse proxy tagged every agent step by model and tool. The team set a 4-step hard cap on the planner loop and routed summarization to a smaller model. Variance dropped from 15x to under 2x.

Lever: Model-router attribution + hard step budget

worked example

Customer support bots: $1,200/week in ghost retries

A startup's support bot silently re-sent the same context-heavy prompt every time a downstream API returned a 502. The retries were not visible in the provider dashboard because each call used a fresh thread ID.

before

$4,800 / month

after

$900 / month

the fix

Synapse flagged repeated near-identical prompts within a 60-second window and blocked the retry storm. Engineering got a Slack alert with the exact request fingerprint and the culprit endpoint.

Lever: Recursive-loop detection + anomaly alerts

worked example

RAG pipeline: context bloat doubled retrieval costs

An enterprise RAG system embedded entire conversation threads plus every retrieved document chunk into every follow-up call. Token counts grew linearly with chat history, turning a cheap retrieval step into the biggest line item.

before

$22,000 / month

after

$9,500 / month

the fix

The Synapse proxy stripped redundant history, deduped retrieved chunks, and cached the system prompt at the edge. The result was a 57% reduction in dispatched tokens with no drop in answer accuracy.

Lever: Dynamic context pruning + prompt caching

get your own audit

Think you have ghost spend?

Share your stack and a recent bill spike. We'll trace the most likely culprits and show you the exact proxy rules that would catch them.

Request a cost audit Read the optimization playbook