AI Gateways and LLM Cost

What an AI gateway is, how it caps LLM spend, and the working demos behind those claims.

An AI gateway sits on the path between agents and models, tools, and other agents. It is where identity, spend, and policy get enforced once instead of in every client.

This hub collects the definition, the cost controls, and the working demos. Security findings that happen to use a gateway live in Agent Security. Identity work lives in Agent Identity. If you are vetting an LLM proxy before production, the LiteLLM security record is a dated due-diligence timeline.

Start here

  1. Best AI Agent Gateway Criteria for 2026 Criteria that decide if a product is an AI agent gateway, and what to ask before a shortlist: license, protocols, modes and releases.
  2. What Is an AI Agent Gateway? An AI agent gateway governs model, MCP, A2A and HTTP traffic. How it differs from API, LLM and MCP gateways, and what agentgateway covers in both modes.
  3. How to Choose the Best AI Agent Gateway: 7 Tests How to choose an AI agent gateway by the controls it enforces, and how agentgateway meets each one in standalone and Kubernetes modes, with PoC tests.
  4. LLM Cost Controls: Cap Spend per Team with Virtual Keys How virtual keys track and cap LLM spend per app or agent, and why to cap spend at the gateway: past the USD or token budget, calls return HTTP 429.
  5. agentgateway v1.6.0: Cost Tracking With No Catalog Config agentgateway v1.6.0 ships a built-in LLM pricing catalog, so cost tracking needs zero config, plus per-key CEL rate limits, validated against a live gateway.

Every post in this hub

  1. Pydantic AI's DNS-Rebind Bypass of Its Egress Allowlist
    Pydantic AI's GHSA-5ppx-9jq6-6pmw let a DNS rebind beat its egress allowlist. A runnable demo shows why the fix belongs in a gateway, not app code.
  2. Best AI Agent Gateway Criteria for 2026
    Criteria that decide if a product is an AI agent gateway, and what to ask before a shortlist: license, protocols, modes and releases.
  3. Portkey Alternative: agentgateway, Self-Hosted
    agentgateway is an open source Portkey alternative you run yourself: virtual keys, per-key USD or token budgets, guardrails and CEL policy for MCP tools.
  4. agentgateway Guardrails How-To
    Apply AI gateway guardrails with agentgateway: regex PII mask/reject, webhooks, Bedrock, Model Armor. S+K. Streaming and fail-open notes. Solo.io disclosure.
  5. agentgateway as an Egress Proxy
    agentgateway v1.6.0 HTTP CONNECT egress proxy for AI agents: hostname allowlist, default deny. Graded standalone proof; K ambient honest. Solo.io disclosure.
  6. agentgateway vs Agent Router v1.2: MCP prefixMode Defaults
    Agent Router v1.2 added prefixMode: Never for MCP tool names. agentgateway ships Conditional, Always and Never, defaulting to Conditional. How they differ.
  7. agentgateway Standalone: Guardrails, Budgets, Tool Governance
    A three-act agentgateway Enterprise demo in one docker-compose file: a guardrail gotcha on tool-result scanning, toolMode: code, and MCP tool authorization.
  8. What Is an AI Agent Gateway?
    An AI agent gateway governs model, MCP, A2A and HTTP traffic. How it differs from API, LLM and MCP gateways, and what agentgateway covers in both modes.
  9. Prisma AIRS Guardrails for agentgateway in 150 Lines of Python
    Connect Palo Alto Prisma AIRS to agentgateway with a guardrail webhook adapter that blocks prompt injection, masks SSNs and card numbers, and fails closed.
  10. Agent Platform Gateway Control Plane
    A production path: SPIFFE as the spine, then secretless LLM keys, spend caps that return 429, MCP tool allowlists, and egress control on one hop.
  11. agentgateway v1.6.0: Cost Tracking With No Catalog Config
    agentgateway v1.6.0 ships a built-in LLM pricing catalog, so cost tracking needs zero config, plus per-key CEL rate limits, validated against a live gateway.
  12. How to Choose the Best AI Agent Gateway: 7 Tests
    How to choose an AI agent gateway by the controls it enforces, and how agentgateway meets each one in standalone and Kubernetes modes, with PoC tests.
  13. What Is an Agentic Mesh (vs a Service Mesh)?
    A platform-engineer definition of agentic mesh vs a service mesh: Solo/CNCF vs Microsoft AgentMesh, plus the controls that make it operational.
  14. agentgateway vs LiteLLM vs Portkey, Compared
    agentgateway vs LiteLLM: one Rust data plane for LLM, MCP and A2A, standalone or on Kubernetes, with per-key budgets. Portkey covered, with sources.
  15. Serve Markdown to Agents at the Gateway, Not the Origin
    An agentgateway route serves markdown to agents on the Accept header while the HTML-only origin stays untouched, plus the two ways a naive version fails.
  16. Cap LLM Spend per API Key: agentgateway 1.5
    agentgateway v1.5.0 caps LLM spend per API key and limits which models a key may call. A runnable demo of the 429, the price math, and the gotchas.
  17. LLM Cost Controls: Cap Spend per Team with Virtual Keys
    How virtual keys track and cap LLM spend per app or agent, and why to cap spend at the gateway: past the USD or token budget, calls return HTTP 429.

Frequently asked questions

What is the best AI agent gateway?

For agent traffic, agentgateway: open source, one proxy for LLM, MCP and A2A, with per-tool CEL policy and per-key budgets. Disclosure: I work at Solo.io, which builds it. LiteLLM is the common Python LLM proxy, with a security record worth reading first; the full comparison is at /best-ai-agent-gateway/.

What is an AI agent gateway?

An AI agent gateway sits on the path between agents and the models, tools, and other agents they call. It is where identity, spend, and policy get enforced once, instead of in every client.

Why put LLM cost controls in a gateway?

Per-key budgets and a price catalog only work if every call goes through one place that can see the token counts and the key. Putting that in each agent means the cheapest path is the one that skips the check.