The Hidden Bill: Where AI Agent Costs Actually Come From

title: "The Hidden Bill: Where AI Agent Costs Actually Come From"
published: true
description: "The model's per-token price is the cheapest part of running an AI agent. The real bill comes from the loop around it. Here's where the money actually goes."

tags: ai, agents, llm, cost

Everyone budgets AI agents the same wrong way: look up the model price ("$3 per million tokens, cheap"), multiply by how many questions you'll ask, feel good.

Then the bill lands at 5–10x that.

The per-token price was never the number that mattered. An agent isn't a chatbot you ask once — it's a loop that re-reads, retries, calls tools, and thinks in steps, and every one of those is billed. Here's where the money actually goes, with real September 2026 prices.

The number everyone quotes (and why it lies)

Current API pricing per million tokens, input / output:

You reason: "2,000-token prompt, 500-token answer — under 2 cents on Sonnet. I could run thousands for pocket change."

Correct for a single call. Completely wrong for an agent. Here's why.

Leak #1: You pay for the whole conversation, every single turn

This is the one that surprises people most, so start here.

LLMs are stateless. The model doesn't "remember" your conversation — every turn, your client re-sends the entire history as input. Turn 10 isn't billed as one message. It's billed as all 10 messages, plus every tool result in between.

So an 8-step task doesn't cost 8 × one call. It's closer to 1 + 2 + 3 + … + 8 — the context grows every turn and you pay for all of it, again, each turn.

A coding agent finishes in 12 turns. By turn 12 the context is 40,000 tokens: the task, the files it read, past tool outputs, its own reasoning. You didn't pay for 40K once — you paid a growing slice on every turn to get there. Re-sent input quietly becomes the biggest line item, often bigger than all your output combined.

The wrong instinct: "input is cheap, output is what costs." True for a chatbot. For an agent, re-sent input usually tops the bill.

Leak #2: Tool calls are round trips, and each one restarts the meter

Every time your agent calls a tool — read a file, hit an API, run a search — that's not free thinking time. It's a full round trip:

1. Model outputs "I want to call tool X" (billed output).
2. Your code runs the tool.
3. The tool's result gets appended to the context and sent back (billed input, forever, on every later turn too).

A big tool result is the worst offender. Dump a 10,000-token API response or file into the context, and you now pay to re-send those 10,000 tokens on every subsequent turn of the loop. One fat tool output can cost more than the entire rest of the task.

The fix people miss: trim tool outputs before they enter context. You rarely need the whole JSON blob — you need three fields. Summarize or filter tool results before they hit the model, not after.

Leak #3: Retries and dead ends you never see

In a demo, the agent nails the task in a clean line. In production, it doesn't.

• It calls a tool with a bad argument, gets an error, tries again.
• It goes down a wrong path for four turns, realizes, and backtracks.
• Your framework retries on a timeout or a malformed response.
• A guardrail rejects an action and the agent re-plans.

Every one of those is billed at full price, and none of them show up in your happy-path math. A task you budgeted at 5 turns routinely runs 9 in the wild. That's not a bug — that's what "agentic" means. Budget for the wandering, not the demo.

Leak #4: Multi-agent fan-out multiplies everything

The moment you go from one agent to "a planner that spawns sub-agents," your costs don't add — they multiply.

Each sub-agent has its own context, its own loop, its own tool calls, its own retries. A planner that fans out to five workers isn't 5x one call; it's five independent instances of Leaks #1–#3, plus the planner's own overhead synthesizing their results. Fan-out is a fantastic capability and a fantastic way to 10x a bill without noticing.

Rule of thumb: before you add a sub-agent, ask if a single agent with one more tool would do. Parallelism is worth paying for when the wall-clock time matters — not by default.

Leak #5: The expensive model doing cheap work

Teams pick one model — usually a strong, pricey one — and route everything through it. But most of what an agent does is not hard:

• classifying which path to take
• extracting a value from text
• deciding "is this done yet?"
• formatting output

That's Haiku work ($1/$5) being billed at Opus rates ($5/$25) — a 5x markup for tasks the cheap model does just as well. The reasoning-heavy step might genuinely need the big model. The other eleven steps in the loop usually don't.

The levers that actually move the bill

Now the good news. Once you see the loop, the fixes are obvious and most of them are free wins, not tradeoffs:

• Prompt caching. This is the big one. Cached input reads cost ~0.1x the base rate — a 90% cut on the repeated part of your context. Since agents re-send the same system prompt and early context every turn (Leak #1), caching is aimed directly at your biggest line item. On Sonnet, cached reads drop from $3 to $0.30 per million. Turn it on before anything else.
• Trim tool outputs before they enter context (kills Leak #2).
• Cap the loop. A hard turn limit stops a wandering agent from billing you for its confusion (Leak #3).
• Route by difficulty. Cheap model for the routing/extraction/"are we done" steps, expensive model only for the hard reasoning (Leak #5).
• Batch the non-urgent work. Batch processing is 50% cheaper across the board. If it doesn't need to be real-time, batch it.

Stack caching + batching and effective spend can fall 90%+ on the repeated parts of your workload. Not by using a worse model — by not paying full price for the same tokens over and over.

The mental model to keep

Stop pricing agents like chatbots. A chatbot is one call. An agent is a loop that re-sends a growing context, pays for every tool round trip, wanders when it's confused, and multiplies when it fans out.

The model's sticker price is the cheapest variable in that whole system. The bill is written by the loop.

Cost the loop, not the token — and cache the part you keep re-sending.

What's the biggest surprise you've hit on an agent bill? I'm especially curious whether re-sent context or tool-output bloat was the bigger leak for you — I keep seeing teams blame the model when it was the loop the whole time. 👇