How to limit LLM usage: quotas that don’t break the product

The bill is almost never a single culprit. It’s a multiplier. One slightly-too-long prompt becomes five calls, then twenty because a tool is flaky, and now you’re paying for the agent’s persistence. If you only “optimize prompts”, you’ll still get surprised—just a little later.

• Look for multipliers first: retries, loops, parallel calls, and context growth.
• Separate “useful tokens” from “panic tokens” (the ones you spend while the system is already failing).
• Budget at a level you can act on: per agent / per workflow / per key—not only a monthly account cap.

If you need something you can ship this week, do it in this order. It’s not glamorous, but it works.

• Instrument first: log tokens + cost per request, plus model, key, agent, tool name, and retry count.
• Add caps: per-request max tokens, per-agent budget, and a hard timeout per run.
• Fix the top offenders: in practice it’s often one retry loop, one context bloat issue, and one overly-chatty tool step.
• Optimize last: prompt tightening, caching, and model selection shine after the system stops spiraling.

The goal isn’t to make every request cheap. The goal is to make the expensive requests predictable—and clearly worth it.

• Budget burn rate: “this agent will exhaust today’s budget in 12 minutes.”
• Retry density: retries per minute (not just total retries).
• Context slope: tokens per turn increasing steadily across the run.
• Tool churn: the same tool called repeatedly with near-identical inputs.
• Cost spikes after failures: spend rising while success rate drops.

• No owner for spend: cost is ‘infra’ until it becomes a fire.
• One giant key for everything: you lose per-agent accountability and can’t rotate safely.
• Retries without a backoff or cap: it looks resilient, then eats your budget under outage.
• Unlimited context growth: a small memory feature quietly becomes a cost monster.
• Only tracking tokens, not outcomes: cost per successful task is the metric that wins arguments.