Skip to content
IQ Routing

Billing

Every request is split into four token classes: input, output, cached, and thinking. Each class has its own per-provider rate; the four-way split lets you see exactly which class is driving spend on the usage page. This page covers the four classes, how cached tokens actually bill, how thinking budgets resolve, the per-team budgets, and the failure signals you get when a limit or a provider account runs out.

Token classes

  • Input tokens are the prompt tokens billed by the provider.
  • Output tokens are the model's response tokens.
  • Cached tokens are input tokens the destination provider served out of its own prompt cache. They are billed, at the provider's discounted cache-read rate, not at zero. See below.
  • Thinking tokens are the reasoning tokens emitted by Anthropic's thinking mode, OpenAI's reasoning models, and Gemini's thinking models. They are billed at the thinking rate, which can be ten to a hundred times the output rate.

What "cached" costs

Two different things produce a cached number on your bill, and they bill in opposite ways, so it is worth being precise about which one you are looking at.

A gateway response-cache hit is billed at zero. When IQ serves a request out of its own response cache it never calls the provider at all, so there is no upstream cost to pass through. It still counts against your rate limits and against the pre-call daily-budget encumbrance, because both run ahead of the cache lookup. A measured production hit returned in about 11 ms at no cost.

A provider prompt-cache read is a different animal. The request did go upstream, the provider recognised a prefix it had already stored, and it charged you its discounted cache-read rate for those tokens. That rate is substantially below the uncached input rate, but it is not free. The other half of that trade is the write side: seeding a provider prompt cache costs more per token than an uncached input token would have, so a prefix that is written often and read rarely loses money. The dashboard now reports the read and write sides separately on each request's detail page, so whether your prompt-cache strategy pays for itself is something you can inspect rather than infer. As a reference point from production, a cached repeat request billed about 18 percent of what its uncached twin cost.

For the mechanics of the gateway's own response cache, including the exact and semantic layers, conversation-scoped caching for agent loops, and streaming replay, see caching.

Thinking budgets

A request's thinking budget resolves to one of four settings: none, low, medium, or high. Medium is the newest tier and fills the gap between a cheap nudge and a full reasoning pass, which previously meant choosing between spending almost nothing on reasoning and spending a lot. Each setting maps onto whatever the destination provider actually supports, so the same choice becomes an Anthropic thinking budget, an OpenAI reasoning effort, or a Gemini thinking budget without you changing anything at the call site. That is the point of expressing it as a tier rather than a provider-specific number: routing can move a request across providers and the intent survives the move.

There is also an org-level auto mode. With auto on, the gateway's complexity judge picks the tier per request rather than the caller pinning one, which is usually what you want for mixed traffic where the caller cannot know in advance whether a given prompt deserves a reasoning pass. Every request records where its budget came from, whether the client asked for it, the org default applied, or auto derived it, so the choice stays auditable on the usage page after the fact.

Budgets

Daily spend caps are per team; the monthly cap is per account. Once a cap is hit the gateway returns 429 until the bucket resets: the daily bucket at midnight UTC, the monthly one at the start of the next month. A per-key daily cap overrides the team cap when it is set tighter.

Every 429 carries X-RateLimit-Scope naming which limit actually fired. Scope values in production are rpm, tpm, daily_budget, monthly_budget, and key_spend_ceiling. The backoff hints ride along only where there is a clock to name: Retry-After plus X-RateLimit-Reset on rpm and tpm, and Retry-After alone on daily_budget, pointing at the next UTC midnight. Do not write a backoff loop that assumes a reset hint is always there. The scope header is the useful piece: a client can tell a request-rate burst apart from a blown monthly budget from the response alone, without opening a support ticket to find out which knob it hit.

A per-request spend ceiling on cap:orchestrate routes is a separate control from the budgets above: it checks a single request's worst-case cost estimate before dispatch rather than a rolling per-team or per-account bucket, and a breach returns a 4xx naming the estimate and the cap rather than a 429 with an X-RateLimit-Scope value. See capability aliases for the mechanics.

Provider credit exhaustion

A 429 means one of your own IQ-side caps fired. A 402 means something else entirely: the upstream provider account is dry.

When every model in the fallback ladder fails because your own provider key has run out of credit, the gateway returns 402 with an actionable message naming the dead provider, of the form Provider credit exhausted on your BYOK key for <provider>; add credits with your provider or configure another provider. It also raises a billing alert in the dashboard's live activity feed, so an operator sees it in the surface they already watch rather than by reading logs after the fact.

Every plan is BYOK, so the provider account that can run dry is always your own. There is no IQ-side pool of provider credit sitting in front of it, which is why the 402 can name the dead provider rather than falling back to a generic upstream failure.

Thinking-budget report

The thinking class is the key FinOps lever. A complex request to a thinking model can spend ten to a hundred times the output-token cost of an equivalent non-thinking request. The thinking-budget report surfaces the per-team and per-complexity-tier aggregate so an admin can spot which team's complex traffic is driving the bulk of monthly spend.

Note that this report groups by the complexity tier the router scored the request at, which is a different axis from the four thinking budget settings above. A request scored complex may still have run with a low budget if the caller pinned one, and a request scored simple under auto mode will normally have drawn none or low. Read the report as where your spend concentrates by workload shape, and the per-request budget source on the usage page as why any single request cost what it did.

The report groups spend by team and then by complexity tier, where the tier is one of simple, medium, or complex.

Endpoint

GET /admin/billing/thinking-report
    ?org_id=<uuid>
    &period_start=2026-04-01
    &period_end=2026-04-30
Authorization: Bearer <session token>

Requires owner or admin role. Cross-org probes return 404.

A sample response:

{
  "org_id": "org_5d9f...",
  "period_start": "2026-04-01",
  "period_end":   "2026-04-30",
  "total_tokens": 18470000,
  "total_cost_usd": 924.50,
  "by_team": [
    {
      "team_id": "team_a1b2...",
      "team_name": "Research",
      "total_tokens": 14820000,
      "total_cost_usd": 741.00,
      "by_tier": [
        {"tier": "simple",  "tokens":   320000, "cost_usd":   16.00},
        {"tier": "medium",  "tokens":  4500000, "cost_usd":  225.00},
        {"tier": "complex", "tokens": 10000000, "cost_usd":  500.00}
      ]
    },
    {
      "team_id": "team_c3d4...",
      "team_name": "Support",
      "total_tokens": 3650000,
      "total_cost_usd": 183.50,
      "by_tier": [
        {"tier": "simple", "tokens": 2400000, "cost_usd":  60.00},
        {"tier": "medium", "tokens": 1250000, "cost_usd": 123.50}
      ]
    }
  ]
}

Dashboard

The Settings → Billing pane carries a thinking-budget report tile rendered as a stacked bar chart. Each bar is one team; the bar segments are the three tier buckets (simple, medium, complex). Hovering a segment shows per-team-per-tier token count and dollar cost.