Every agent platform demo eventually meets the same question from the person who signs off on it: who is spending what, and what stops a runaway agent from spending more? I added that layer to my Solo agentic stack demo as an eighth act. It gives every LLM request a price, attributes it to a user and a team, and refuses the request with an HTTP 429 once a budget is gone. All of it lives at the gateway, which is the only place you can do it once for every agent, every framework, and every provider at the same time.

The manifests are in manifests/cost-management/ and the walkthrough is ./demo.sh --act 8.

Why token counts are not costs

Gateways have counted tokens for a long time. Counting is not costing. Without prices, every spend number in your dashboard is zero, and “we used 4.1 billion tokens last month” is not a number anyone can act on.

Three things have to line up before a dollar figure appears, and the important part is that each one fails quietly rather than loudly:

  1. Prices. A per-model price catalog attached to the Gateway.
  2. Attribution. A key that says which user and which team a request belongs to.
  3. Enforcement. A budget that reads the first two and blocks when exceeded.

Miss the first and you get traffic charts with zero dollars. Miss the second and everything lands in a bucket called Unattributed. Neither throws an error.

Loading a model price catalog

The catalog is JSON keyed by provider, then model, with rates as exact decimal strings in USD per million tokens. Strings, not floats, so nothing gets rounded on the way in.

apiVersion: v1
kind: ConfigMap
metadata:
  name: agw-model-costs
  namespace: agentgateway-system
data:
  catalog.json: |
    {
      "providers": {
        "anthropic": {
          "models": {
            "claude-sonnet-4-6": {
              "rates": { "input": "3.00", "output": "15.00", "cacheRead": "0.30" }
            }
          }
        },
        "openai": {
          "models": {
            "gpt-4o": {
              "rates": { "input": "2.50", "output": "10.00", "cacheRead": "1.25" }
            }
          }
        }
      }
    }

The Gateway picks it up through modelCatalog.sources on the EnterpriseAgentgatewayParameters it already references:

spec:
  modelCatalog:
    sources:
    - configMap:
        name: agw-model-costs
        key: catalog.json

Order matters. Apply the ConfigMap before the parameters that point at it, or the catalog only takes effect on a later reconcile. Once loaded, the gateway populates a llm.cost field on every request that flows into access logs, traces, and metrics.

Hand-maintaining prices is a bad idea long term. agctl costs import --providers openai,anthropic generates the catalog from shipped price data.

Virtual keys, and how attribution actually resolves

A virtual key is a gateway-issued token carrying attribution metadata. The caller sends one key, the gateway resolves it into cost dimensions, and the real provider credential never leaves the gateway.

apiVersion: v1
kind: Secret
metadata:
  name: llm-virtual-keys
  namespace: agentgateway-system
  labels:
    agentgateway.solo.io/virtual-key-set: demo
type: Opaque
stringData:
  alice: |
    {
      "key": "sk-alice-demo",
      "metadata": {
        "id": "alice",
        "user": "alice",
        "group": "platform-eng"
      }
    }

The mapping from that metadata to budget dimensions is worth writing down, because it is not obvious from the resource:

  • virtualKey resolves from apiKey.id, which is metadata.id above.
  • user resolves from a coalesce over apiKey.user, apiKey.name, apiKey.owner, jwt.sub, jwt.email and more.
  • group resolves from coalesce(jwt.group, apiKey.group).
  • model and provider are built in and need no expression.

Those defaults live in the chart’s budgetDimensions.config. Custom dimensions (cost centre, tenant) are CEL expressions added to the same block.

The policy that consumes the key set uses mode: Strict, which is a deliberate choice. Strict means a valid key is required, so on this route unattributed spend cannot happen at all rather than merely being visible:

  traffic:
    apiKeyAuthentication:
      mode: Strict
      secretSelector:
        matchLabels:
          agentgateway.solo.io/virtual-key-set: demo
    entBudgetEnforcement: {}

entBudgetEnforcement is the switch that makes budgets apply to this route. Without it the budgets exist and do nothing.

Writing budgets in tokens and dollars

One EnterpriseAgentgatewayBudget holds up to 64 independent entries. Each is a subject, a limit, a window, and an action.

apiVersion: enterpriseagentgateway.solo.io/v1alpha1
kind: EnterpriseAgentgatewayBudget
metadata:
  name: cost-demo-budgets
  namespace: agentgateway-system
spec:
  budgets:
  - name: per-key-daily-tokens
    subject:
      virtualKey: "*"
    limit:
      unit: Tokens
      amount: 100
    window:
      unit: Day
    onBudgetExceeded: Block

  - name: platform-eng-monthly-usd
    subject:
      group: platform-eng
    limit:
      unit: USD
      amount: 50
    window:
      unit: Month
    onBudgetExceeded: Audit

subject.virtualKey: "*" gives each key its own bucket, so one caller running dry has no effect on another. Block returns a 429. Audit records the breach and lets the request through, which is how you roll budgets out without causing an outage on day one. Windows are rolling from the first debit, not calendar aligned.

USD limits are whole numbers, so one dollar is the smallest budget you can write. Dollar budgets also depend entirely on the catalog: a model the catalog cannot price adds zero, so an incomplete catalog undercounts silently.

The controller confirms it took the resource:

kubectl get eagbud cost-demo-budgets -n agentgateway-system \
  -o jsonpath='{.status.conditions[*].reason}'
# Valid Enforced

Proving enforcement with a 429

This is the part worth showing live. Six requests succeed, the seventh does not, and a second key on its own bucket is untouched.

alice request 1:    HTTP 200
alice request 2:    HTTP 200
alice request 3:    HTTP 200
alice request 4:    HTTP 200
alice request 5:    HTTP 200
alice request 6:    HTTP 200
alice request 7:    HTTP 429

bob (own bucket):   HTTP 200

A 100 token daily budget against gpt-4o with max_tokens: 20 works out to roughly 17 tokens per “hi” (8 in, 9 out), so the flip lands on the seventh request. My first version of the demo loop ran six requests and stopped, which showed six 200s and no 429, the exact opposite of the point. Count your own token cost and make the loop run past it.

Checking that pricing actually landed

The failure mode here is silence, so verify with the counter rather than trusting the dashboard:

kubectl port-forward deploy/agentgateway-proxy -n agentgateway-system 15020:15020 &
curl -s localhost:15020/metrics | grep agentgateway_cost_catalog_lookups_total

agentgateway_cost_catalog_lookups_total{status="Exact",gen_ai_system="openai",
gen_ai_request_model="gpt-4o",gen_ai_response_model="gpt-4o-2024-08-06",
route="agentgateway-system/metered-llm"} 2

status="Exact" means the request was priced from the catalog. Missing, Unpriced, or NoCatalog mean spend is being undercounted, and nothing anywhere will tell you that except this counter.

Two caveats worth saying before someone asks

Both of these are properties of the design, and a FinOps buyer respects hearing them up front more than a claim of a hard cutoff.

Enforcement is approximate. Token counts are not known until the response comes back, so usage is debited after the fact. A burst can overshoot the limit slightly before the next request is refused.

Budgets fail open. If the rate limit service is unreachable, requests are allowed through. Availability is chosen over a hard spend cutoff.

Why budgets never trip when a provider is down

Budgets debit usage from the response. A provider that refuses the request produces no usage, so nothing is debited and the budget never trips. In my case one provider account was out of credit and the gateway faithfully passed the error back:

{"error":{"type":"invalid_request_error",
  "message":"Your credit balance is too low to access the Anthropic API."}}

The gateway did its job. Authentication passed, the request was priced, the upstream was called. But the whole cost-control act rendered as a row of 400s instead of the 429 it exists to show. I pointed the metered route at the provider that was actually answering, which also insulates the act from any single provider’s billing state.

Four silent failures found while bumping versions

Moving the stack to current releases (agentgateway v2026.8.2, kagent Enterprise 0.5.5, AgentRegistry 2026.8.0) surfaced a pattern: none of these failed loudly.

Helm accepts unknown --set keys without complaint. kagent 0.5.x renamed the built-in agent toggle from agents.k8s-agent.enabled to k8s-agent.enabled. The old key produced no error, just a missing agent. helm template settles it in seconds:

helm template k ./kagent-enterprise --set k8s-agent.enabled=true | grep -c "^kind: Agent"
# 1
helm template k ./kagent-enterprise --set agents.k8s-agent.enabled=true | grep -c "^kind: Agent"
# 0

A stale pin in .env silently overrides the script. My setup script uses ${VAR:-default} and sources .env first, so an old AGW_VERSION in a gitignored file quietly won every version bump I made to the script. Worth checking before concluding a bump did not work.

kubectl wait does not wait for objects that do not exist yet. The mesh readiness check looked patient and was not:

Error from server (NotFound): error when creating "istiod-alias.yaml":
namespaces "istio-system" not found

Against a namespace that has not been created, kubectl wait fails in milliseconds rather than blocking, so the advertised 300 second timeout was never spent. The script raced ahead into a fatal apply and died under set -e. The fix is to poll for existence first, then wait on conditions.

A new required field breaks one object and not the rest. AgentRegistry 2026.7.x began requiring config.auth.oidc on kagent runtimes:

✗ Runtime/kagent failed: prepare: invalid input: auth.oidc is required for kagent runtimes
✓ MCPServer/weather-mcp (latest) created
✓ Agent/weather-assistant (latest) created

Everything else registered fine. The catalog looks healthy right up to the moment you try to deploy an agent and discover there is no runtime to deploy it onto.

Running it

git clone https://github.com/themsquared/agentic-demo
cd agentic-demo
cp .env.example .env        # add LLM keys and a Solo license
./setup.sh                  # cluster, mesh, gateway, kagent, registry
./port-forward.sh
./demo.sh --act 8           # cost management, live

Act 8 applies the same files in manifests/cost-management/ that this post quotes, so the YAML on screen is the YAML that runs.

What is next

The dimensions model is the interesting thread. Cost centre and tenant are CEL expressions away, which makes chargeback per team a config change rather than a data pipeline. I am also curious whether budgets can key off an agent identity rather than a human one, since in an agentic mesh the caller is usually another agent.

If you want the layer underneath this, I wrote up running thousands of agents on tens of pods with kagent Agent Substrate. The code for everything here is at themsquared/agentic-demo.