Cost, Usage & Resource Tracking

Agent costs are odd. They depend on use, they come in spikes, and they follow behavior you do not fully control. A small prompt change, a looping bug or one very chatty customer can change the bill by ten times overnight. Cost and usage operations means three things. Measure exactly who and what uses resources. Control it with budgets and good design. And link the spend to the value it produces.

Meter everything, assign everything

Your provider's invoice gives you the total. It does not tell you which feature, customer or prompt caused it. Record a usage event for every model call, with enough labels to slice the spend later.

CREATE TABLE usage_events (
  id              TEXT PRIMARY KEY,
  occurred_at     TEXT NOT NULL,
  tenant_id       TEXT NOT NULL,       -- who pays
  user_id         TEXT,
  feature         TEXT NOT NULL,       -- e.g. 'support-triage', 'report-writer'
  agent           TEXT NOT NULL,       -- which agent/sub-agent
  run_id          TEXT NOT NULL,       -- ties to the trace
  model           TEXT NOT NULL,       -- exact snapshot
  input_tokens    INTEGER NOT NULL,
  cached_tokens   INTEGER NOT NULL DEFAULT 0,
  output_tokens   INTEGER NOT NULL,
  cost_micros     INTEGER NOT NULL,    -- USD * 1e6, avoids float drift
  outcome         TEXT                 -- 'success' | 'fail' | 'escalated' (filled in later)
);
CREATE INDEX usage_tenant_time ON usage_events (tenant_id, occurred_at);

-- The questions this table must answer in one query each:
--  * cost per successful task, by feature
--  * top 10 tenants by spend this week, and their margin
--  * which agent in the pipeline is the expensive one
--  * did last Tuesday's prompt change move tokens per run?

Unit economics: cost per successful outcome

Turn tokens into business terms. Cost per successful task is the total spend divided by the tasks that really succeeded. Compare it with what the task is worth, such as a resolved ticket that saves 8 minutes of agent time, a qualified lead or a delivered report. Compare it with what you charge too. Track the gross margin for each customer or plan. If heavy users can cost more than they pay, you need pricing by use, fair-use limits or a cheaper design. It is far better to learn that from a dashboard than from the invoice.

Budgets, quotas and circuit breakers

• A budget for each run: the most tokens, steps and dollars for one run.
• Quotas for each user and customer: daily or monthly caps that degrade gently, with a smaller model and shorter output, before they stop hard.
• A global spending ceiling: an automatic circuit breaker that pauses non-critical agents if hourly spend goes over N times the recent average.
• Alerts on speed and oddities. A sudden 5 times jump in tokens per run is a bug until proven otherwise.
• Separate API keys for each environment and each major feature. Then a runaway test script cannot eat the live budget, and it is easy to see who spent what.

def check_quota(tenant_id: str, estimated_cost_micros: int) -> str:
    spent = usage.sum_cost(tenant_id, since=month_start())
    plan  = plans.for_tenant(tenant_id)

    if spent + estimated_cost_micros > plan.hard_cap_micros:
        return "deny"               # tell the user plainly, offer upgrade
    if spent > plan.soft_cap_micros:
        return "degrade"            # route to cheaper model, cap output length
    return "allow"

# Global breaker (cron or stream processor):
# if spend_last_hour > 5 * trailing_7d_hourly_mean: pause_noncritical_agents(); page("spend spike")

Ways to cut costs, with real teeth

• Model routing: send easy requests to small models. Measure the quality cost with your test set before you route.
• Prompt caching: put the stable part first and the changing part last. Check the cache-hit rate in your usage events.
• Context control: cut tool results short, shrink the history and do not carry everything forward.
• Output control: limit length and format. Structured outputs are shorter than prose.
• Response caches, exact or by meaning, for repeated questions where a slightly old answer is fine.
• Batch or background processing for work that does not need an instant answer.
• Stop zombie loops with step limits and a check for no progress.

Provider limits and capacity

Cost is not the only meter. Providers limit requests and tokens per minute, and some regions run short of capacity. Track how close you are to the limits. Ask for higher limits before launches. Queue and smooth out bursts of work. And design a fallback, so that hitting a limit reduces the service instead of breaking it. The reliability lesson covers this.

Resource footprint: energy, water and carbon

More and more customers and regulators ask what AI use costs the planet. Estimates of energy, water and CO₂ for each request are rough. They vary widely between sources, often by ten times or more, depending on the model, the hardware, the data center and the power grid. Only a few providers have published measured figures. That does not mean you should ignore it. Your usage ledger already records tokens by model. So you can multiply by a documented low, middle and high estimate for each model. Show the result as a range with its source. Use it to compare designs. For example, a router that sends 70% of traffic to a small model usually cuts the footprint as well as the cost. The model pages in this product show that style of sourced, banded estimate. Never show one precise number as if it were a measurement.