AI Gateways and LLM Cost

What an AI gateway is, how it caps LLM spend, and the working demos behind those claims.

An AI gateway sits on the path between agents and models, tools, and other agents. It is where identity, spend, and policy get enforced once instead of in every client.

This hub collects the definition, the cost controls, and the working demos. Security findings that happen to use a gateway live in Agent Security . Identity work lives in Agent Identity .

Start here

  1. What Is an AI Agent Gateway? An AI agent gateway governs model, MCP, A2A and HTTP traffic. How it differs from API, LLM and MCP gateways, and what agentgateway covers in both modes.
  2. Choose an AI Agent Gateway: agentgateway Guide How to choose an AI agent gateway by the controls it enforces, and how agentgateway meets each one in standalone and Kubernetes modes, with PoC tests.
  3. Cap LLM Spend per Team with Virtual Keys Enterprises should cap LLM spend at the gateway. Virtual keys attribute each team, and a monthly budget returns HTTP 429 when the USD or token limit is gone.
  4. agentgateway v1.6.0: Cost Tracking With No Catalog Config agentgateway v1.6.0 ships a built-in LLM pricing catalog, so cost tracking needs zero config, plus per-key CEL rate limits, validated against a live gateway.

Every post in this hub

  1. Portkey Alternative: agentgateway, Self-Hosted
    agentgateway is an open source Portkey alternative you run yourself: virtual keys, per-key USD or token budgets, guardrails and CEL policy for MCP tools.
  2. agentgateway Guardrails How-To
    Apply AI gateway guardrails with agentgateway: regex PII mask/reject, webhooks, Bedrock, Model Armor. S+K. Streaming and fail-open notes. Solo.io disclosure.
  3. agentgateway as an Egress Proxy
    agentgateway v1.6.0 HTTP CONNECT egress proxy for AI agents: hostname allowlist, default deny. Graded standalone proof; K ambient honest. Solo.io disclosure.
  4. agentgateway vs Agent Router v1.2: MCP prefixMode Defaults
    Agent Router v1.2 added prefixMode: Never for MCP tool names. agentgateway ships Conditional, Always and Never, defaulting to Conditional. How they differ.
  5. agentgateway Standalone: Guardrails, Budgets, Tool Governance
    A three-act agentgateway Enterprise demo in one docker-compose file: a guardrail gotcha on tool-result scanning, toolMode: code, and MCP tool authorization.
  6. What Is an AI Agent Gateway?
    An AI agent gateway governs model, MCP, A2A and HTTP traffic. How it differs from API, LLM and MCP gateways, and what agentgateway covers in both modes.
  7. Prisma AIRS Guardrails for agentgateway in 150 Lines of Python
    Connect Palo Alto Prisma AIRS to agentgateway with a guardrail webhook adapter that blocks prompt injection, masks SSNs and card numbers, and fails closed.
  8. Agent Platform Gateway Control Plane
    A production path: SPIFFE as the spine, then secretless LLM keys, spend caps that return 429, MCP tool allowlists, and egress control on one hop.
  9. agentgateway v1.6.0: Cost Tracking With No Catalog Config
    agentgateway v1.6.0 ships a built-in LLM pricing catalog, so cost tracking needs zero config, plus per-key CEL rate limits, validated against a live gateway.
  10. Choose an AI Agent Gateway: agentgateway Guide
    How to choose an AI agent gateway by the controls it enforces, and how agentgateway meets each one in standalone and Kubernetes modes, with PoC tests.
  11. What Is an Agentic Mesh (vs a Service Mesh)?
    A platform-engineer definition of agentic mesh vs a service mesh: Solo/CNCF vs Microsoft AgentMesh, plus the controls that make it operational.
  12. agentgateway vs LiteLLM vs Portkey, Compared
    agentgateway vs LiteLLM: one Rust data plane for LLM, MCP and A2A, standalone or on Kubernetes, with per-key budgets. Portkey covered, with sources.
  13. Serve Markdown to Agents at the Gateway, Not the Origin
    An agentgateway route serves markdown to agents on the Accept header while the HTML-only origin stays untouched, plus the two ways a naive version fails.
  14. Cap LLM Spend per API Key: agentgateway 1.5
    agentgateway v1.5.0 caps LLM spend per API key and limits which models a key may call. A runnable demo of the 429, the price math, and the gotchas.
  15. Cap LLM Spend per Team with Virtual Keys
    Enterprises should cap LLM spend at the gateway. Virtual keys attribute each team, and a monthly budget returns HTTP 429 when the USD or token limit is gone.

Frequently asked questions

What is an AI agent gateway?

An AI agent gateway sits on the path between agents and the models, tools, and other agents they call. It is where identity, spend, and policy get enforced once, instead of in every client.

Why put LLM cost controls in a gateway?

Per-key budgets and a price catalog only work if every call goes through one place that can see the token counts and the key. Putting that in each agent means the cheapest path is the one that skips the check.