agentgateway v1.5.0: Per-Key LLM Budgets That Return 429

agentgateway v1.5.0 shipped on 2026-08-27 with two controls that attach to the API key rather than the route: a rolling spend budget that blocks with an HTTP 429, and a list of models that key is allowed to call. I built themsquared/agw-budget-guard to run both locally against a mock LLM that reports exactly 1000 tokens per call, so the enforcement is provable instead of approximate. No provider API key, no cluster, one docker compose up.

I wrote about pricing and capping LLM spend at the gateway last week using Kubernetes CRDs and a price catalog. This is the standalone counterpart, and the interesting part is not that budgets exist. It is where the enforcement sits.

Why the API key is the right place for a spend cap

A route-level rate limit tells you how many requests per second the platform will accept. It says nothing about which team is spending, and nothing about dollars. A token counter in your dashboard tells you what happened after it happened.

The two questions a platform team actually gets asked are narrower than either:

1. What stops one key from burning the quarter’s budget in an afternoon?
2. What stops a key issued for a cheap model from calling the expensive one?

Both are properties of the credential, not of the path. v1.5.0 puts them there. A key carries a budgets list and an allowedModels list, the gateway charges usage after each response, and the next request over the line gets refused in the data plane before it reaches a provider.

Configuring a budget on an API key

Budgets attach to entries in the apiKey policy on a listener. Each one is a name, a limit with a unit, a rolling window, and what to do on overage:

gateways:
  default:
    port: 3300
    listeners:
    - name: llm
      apiKey:
        mode: strict
        location:
          header:
            name: Authorization
            prefix: "Bearer "
        keys:
        - key: sk-demo-research
          metadata:
            name: research
            team: research
          allowedModels: ["gpt-4o-mini"]
          budgets:
          - name: hourly-tokens
            limit: { unit: Tokens, amount: 5000 }
            window: { rolling: 1h }
            onBudgetExceeded: Block

unit is Tokens or USD. onBudgetExceeded is Block or Audit. Multiple budgets on one key are charged independently, so a key can carry both an hourly token ceiling and a daily dollar ceiling and hit whichever comes first.

The demo repo defines four keys that cover the interesting combinations:

What a blocked request actually looks like

The mock LLM reports 800 prompt tokens and 200 completion tokens on every call, so a 5000-token budget is exactly five calls:

1. Token budget: research is capped at 5000 tokens/hour, 1000 tokens per call
   call 1 -> HTTP 200
   call 2 -> HTTP 200
   call 3 -> HTTP 200
   call 4 -> HTTP 200
   call 5 -> HTTP 200
   call 6 -> HTTP 429

The refusal is shaped like an OpenAI error, so a client SDK’s existing rate-limit handling picks it up without changes:

{"error":{"message":"Budget exceeded","type":"rate_limit_error","code":"budget_exceeded"}}

with a retry-after header carrying the seconds left in the window:

HTTP/1.1 429 Too Many Requests
content-type: application/json
retry-after: 3209

How dollar budgets get their prices

A USD budget needs a price for every model it charges. agentgateway resolves cost through a model catalog, and the demo ships one inline so the arithmetic is reproducible offline:

config:
  modelCatalog:
  - inline:
      providers:
        demo:
          models:
            gpt-4o:
              rates:
                input: "2.50"
                output: "10.00"

At those rates an 800-in / 200-out call against gpt-4o costs $0.002 + $0.002 = $0.004. A $0.01 daily budget therefore survives three calls and refuses the fourth:

5. Dollar budget: platform is capped at $0.01/day, $0.004 per gpt-4o call
   call 1 -> HTTP 200
   call 2 -> HTTP 200
   call 3 -> HTTP 200
   call 4 -> HTTP 429

The counter ends at $0.012, not $0.01. Usage is charged after the response, so a request that starts under the limit is always served and then billed. Budgets are a ceiling on what gets admitted, not a hard cap on what gets spent, and the overshoot is bounded by the cost of one request.

Model allowlists are a separate check with a separate status code

allowedModels takes exact names or a single wildcard at the start or the end of the pattern. Omitting the field means no constraint. An empty list denies everything, which is a useful state for a key that has been provisioned but not yet approved for any model.

The sk-demo-observer key is scoped to gpt-4o*, so gpt-4o and gpt-4o-mini pass and o4-mini does not:

sk-demo-observer -> o4-mini  HTTP 403  {"error":{"message":"Model is not allowed for this API key","type":"invalid_request_error","code":"model_not_allowed"}}
sk-demo-observer -> gpt-4o   HTTP 200

403 with model_not_allowed, distinct from the 429 with budget_exceeded. A client can tell “you cannot afford this” apart from “you were never allowed this” without parsing prose.

The allowlist also shapes discovery. /v1/models returns only what the presenting key may call:

sk-demo-research   ['gpt-4o-mini']
sk-demo-observer   ['gpt-4o-mini', 'gpt-4o']
sk-demo-platform   ['gpt-4o-mini', 'o4-mini', 'gpt-4o']
sk-demo-locked     []

That matters for agents specifically. An agent that lists models and picks one will never see a model it would have been refused, so the failure never has to happen at inference time.

Audit mode records the overage without failing the request

onBudgetExceeded: Audit is the mode to reach for when you want the number before you want the enforcement. The sk-demo-observer key has a 3000-token budget in audit mode, and the walkthrough puts five more calls through it after the allowlist section has already spent one. All of them are served:

4. Audit budgets record the overage and still serve the request
   call 1 -> HTTP 200
   call 2 -> HTTP 200
   call 3 -> HTTP 200
   call 4 -> HTTP 200
   call 5 -> HTTP 200

Six calls total, 6000 tokens against a 3000-token limit, and the counter records every one of them:

{
    "apiKeyName": "observer",
    "name": "hourly-tokens",
    "limit": { "unit": "Tokens", "amount": "3000" },
    "usage": { "used": "6000", "remaining": "0", "exceeded": true },
    "onBudgetExceeded": "Audit"
}

Roll a budget out in Audit, watch the counters for a week, then flip the same config to Block once you know the number is right. That is the migration path for putting a cap on a team that has never had one.

Where the counters live

Budget state is served from the admin interface at /api/budgets/status, optionally filtered by key name:

curl -s 'localhost:15000/api/budgets/status?apiKeyName=research' | python3 -m json.tool

{
    "apiKeyName": "research",
    "name": "hourly-tokens",
    "limit": { "unit": "Tokens", "amount": "5000" },
    "usage": { "used": "5000", "remaining": "0", "exceeded": true },
    "window": {
        "start": 1787932800000,
        "end": 1787936400000,
        "durationMs": 3600000,
        "expired": false
    },
    "onBudgetExceeded": "Block",
    "updatedAt": 1787933179803
}

The state is persisted, not in-memory. Restart the gateway mid-window and the spend is still spent:

docker compose restart agentgateway
# research after restart: HTTP 429

That is the behavior you want and it is worth confirming yourself, because the alternative (a counter that resets on every rollout) turns a budget into a suggestion.

Gotchas

These are the five things that cost me time. Every error message below is verbatim from the v1.5.0 binary.

Budgets need a database, not hybrid storage mode. The release notes say “Budgets require hybrid storage mode.” The requirement the binary enforces is config.database. I ran the demo under the default file storage mode with only a SQLite URL configured and budgets blocked and persisted correctly. Hybrid mode is what lets the UI and admin API write configuration back, which is a related but separate concern. Leaving the database out gives you:

Error: API key budgets require config.database to be configured

A key with a budget needs metadata.name. Counters are keyed by the key’s name, so an unnamed key with a budget is rejected at startup:

Error: API keys with budgets must have a metadata.name

Windows align to the Unix epoch, not to your first request. A 1h window follows UTC clock hours and 24h starts at midnight UTC. This is why the retry-after above is 3209 seconds rather than a round hour: the first request landed partway into the clock hour. If you are reasoning about when a team’s budget resets, it resets on the clock, not on their usage.

The budget check runs before the allowlist check. A key that is both over budget and asking for a model it was never allowed gets 429, not 403. I chased a missing 403 for a few minutes before noticing the key was already exhausted from a previous run.

/v1/models is gated too. Once a blocking budget is exhausted, model discovery returns 429 along with everything else:

sk-demo-research   {'error': {'message': 'Budget exceeded', 'type': 'rate_limit_error', 'code': 'budget_exceeded'}}
sk-demo-observer   ['gpt-4o-mini', 'gpt-4o']

That is defensible (the key cannot call any of them anyway) but it is a different failure surface than most clients expect, and it makes stale budget state from an earlier test run look like a broken config.

A USD budget with no price is silent. A model with no catalog entry still serves traffic, it just never charges the budget. Nothing errors. The only signal is that the counter does not move, so check /api/budgets/status after wiring a new model rather than assuming the cap is live.

Allowlist patterns take one wildcard, at one end. Both of these are rejected at startup:

Error: allowedModels cannot combine '*' with other values
Error: allowedModels pattern "*gpt*" must contain at most one wildcard, at the beginning or end

Run it

Docker and python3 are the only prerequisites.

git clone https://github.com/themsquared/agw-budget-guard.git
cd agw-budget-guard
mkdir -p data
docker compose up -d --build

Then walk through every behavior above:

bash scripts/demo.sh

Or assert them, which is what I run after any config change:

bash scripts/verify.sh

ok    research call 6 blocked (429)
ok    429 body carries code=budget_exceeded
ok    observer o4-mini off allowlist (403)
ok    platform usd call 4 blocked (429)
ok    audit budget recorded 6000 of 3000 tokens used

all checks passed

Budgets persist, so reset between runs with docker compose down && rm -rf data && mkdir -p data. To validate a config edit without starting anything, the v1.5.0 binary takes --validate-only:

MOCK_LLM_URL=http://127.0.0.1:8088 ./agentgateway -f config/agentgateway.yaml --validate-only

What this covers and what it does not

Per-key budgets and allowlists close the two gaps that make an LLM gateway hard to hand to a finance team: attribution that resolves to a credential, and a refusal that happens before the money is spent. The demo proves both offline in about a minute.

What it does not cover is the harder half. Budgets are per key, so the moment agents mint their own credentials you need the issuance path to attach the right budget, and that is a different problem from enforcing one. v1.5.0 also shipped per-request minted JWTs and SPIFFE workload identity, which is where that thread goes next.

The repo is at themsquared/agw-budget-guard. Full config reference for everything above is in the v1.5.0 schema, and the standalone quickstart is the place to start if you want to build this against a real provider instead of a mock.