An AI agent gateway is a reverse proxy at the agent boundary. It governs the traffic agents send to language models, to tools (often over the Model Context Protocol), to other agents (A2A), and to ordinary HTTP or gRPC services. It applies identity, authorization, budgets, audit, and egress on that path. Four labels get used as if they named one box: API gateway, LLM or model gateway, MCP gateway, and AI or agent gateway. They name four jobs. This page defines the category, puts the four labels in one table, and uses open-source agentgateway as the worked example of a unified data plane in both standalone and Kubernetes modes.
Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything below is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project’s own docs or release notes as of 2026-10-05.
This is the definition page. The choose guide is the scorecard. I am not walking the seven controls or the PoC table again here.
What is an AI / agent gateway?
An AI agent gateway sits on the path an agent uses to leave its process. The traffic it sees is not only GET /health. It is a completion, a tools/call, an A2A task, and a normal service request, often on the same hop.
The decisions it is built to make are where the request goes, who may call, what the call costs, what is written down, and what is allowed to leave the network. If a product only routes chat completions, it is doing the LLM-gateway job. If it only fronts MCP servers, it is doing the MCP-gateway job. The agent-gateway label is the broader (or unified) one: several of those traffic types, and the decisions that go with them, in one place.
The Blueprint Alliance , announced 2026-09-22 by Okta with founding members including AWS, CrowdStrike, Databricks, Docker, Google Cloud, Salesforce, ServiceNow, Wiz, Zscaler, and others, describes that hop as runtime authorization “through gateways that sit in the execution path,” including resource access to MCP servers. Okta’s Agent Gateway help defines the same control point as an identity-native proxy that ties identity to every tool call, controls which tools and MCP servers an agent can use, and injects credentials so the agent never holds long-lived API keys. Their runtime-governance post says it “brokers short-lived, isolated tokens dynamically.” Industry framing, not a product pick. The worked example here remains open-source agentgateway, standalone and Kubernetes.
agent --> AI / agent gateway
|
+-----------+-----------+-----------+
| | | |
models MCP A2A peers APIs
The gateway is the hop, not the whole platform. What an agentic mesh is is the wider architecture. This page stays on the gateway category.
Four labels, four jobs (API vs LLM vs MCP vs agent)
Look at traffic and decisions, not at the badge. The same product can wear more than one of these labels.
| Label | Traffic | Typical decisions | What it usually does not cover |
|---|---|---|---|
| API gateway | Ordinary service HTTP or gRPC | Route, authenticate the caller, rate-limit requests, terminate TLS | Tool names inside an MCP body, token or dollar cost of a completion, A2A task state |
| LLM / model gateway | Inference to one or more providers | Which model, fallback, token or dollar cost, prompt controls | Agent-to-tool sessions, per-tool authorization, ordinary service HTTP as the same plane |
| MCP gateway | Agent-to-tool traffic over MCP | Discovery, tools/list filtering, per-tool authorization, sessions, audit | Provider inference routing, a general HTTP/gRPC service plane |
| AI / agent gateway | Several of the above together: models, tools (often MCP), A2A, and often ordinary HTTP/gRPC | Where to send, who may call, what it costs, what is audited, what may egress | Whatever traffic type a given product never parsed. The label does not guarantee the union. |
Two peer taxonomies say the same split in public. Dan Kaplan at Aembit (September 2026) groups offerings into three patterns: LLM or model gateways manage inference; MCP gateways mediate MCP servers and tools; agent gateways handle broader agentic communication. Identity and access cut across all three. That post names open-source agentgateway as an example of the agent-gateway pattern (HTTP/gRPC plus LLM, MCP, and A2A). Kelvin Yu at Ascending (2026-09-16) follows one delegated MCP call and asks which unit the gateway can still see: a route, a caller, a tool catalog, a long-running task. I am citing those taxonomy claims, not either vendor’s product.
Naming: “agent gateway” the category vs agentgateway the project
agentgateway (one word, lowercase) is the open-source project. “Agent gateway” and “AI gateway” are category names. Yu’s Ascending post makes that distinction explicitly: lowercase agentgateway is a Solo-originated Linux Foundation project, not the name of the category. I am keeping the same hygiene here.
The project’s standalone About page (docs labelled 1.6 latest) calls it an AI-first, open-source, cloud-native gateway control plane and proxy data plane, originally donated to the Linux Foundation in 2025 and accepted as an Agentic AI Foundation (AAIF) project in 2026. It is a general-purpose HTTP and gRPC data plane. The same proxy fronts ordinary APIs and microservices, LLM inference, MCP tool servers, and A2A agent traffic. The Kubernetes About page describes the same data plane behind a controller that translates Gateway API and agentgateway custom resources. The site tagline on agentgateway.dev is the union in one line: LLM, MCP, A2A, and HTTP in one data plane.
AAIF acceptance is that documented fact. It is not a popularity rank, a CNCF graduation, or “the standard.”
What agentgateway covers (both modes)
Two official doc trees exist, both returning 200: standalone and Kubernetes . Same proxy. Different install. Every claim below is marked S (standalone), K (Kubernetes), or S+K.
Shared data plane (S+K). HTTP and gRPC, LLM inference, MCP tool traffic, and A2A on one proxy. That is the About claim above, not a second product.
Standalone (S). A single binary plus YAML. It runs on a laptop, a VM, a container, or an edge box. No Kubernetes required. The standalone docs cover the Anthropic Messages API
at /v1/messages, MCP authorization with CEL
, an HTTP CONNECT egress proxy
, and per-key budgets in USD or tokens
.
Kubernetes (K). A Gateway API implementation. The controller translates Gateway API resources plus AgentgatewayPolicy and AgentgatewayBackend over xDS. Open-source Kubernetes budgets are token-based
: global rate limiting with unit: Tokens, rejected with HTTP 429.
| Capability | Mode | Grade |
|---|---|---|
| HTTP, gRPC, LLM, MCP, and A2A on one proxy | S+K | Documented (About ) |
| Single binary plus YAML; no cluster required | S | Documented (standalone ) |
Anthropic Messages at /v1/messages | S | Documented (Messages ) |
| MCP authorization with CEL | S | Documented (MCP authz ) |
| HTTP CONNECT egress proxy | S | Documented (egress proxy ) |
| Per-key budgets in USD or tokens | S | Documented (1.6 latest per-key ); Tested on OSS v1.5.0 standalone in per-key LLM budgets |
Gateway API data plane; AgentgatewayPolicy / AgentgatewayBackend; controller over xDS | K | Documented (K About ) |
| Token-based budgets that return 429 | K | Documented (K budget limits ) |
| Substrate CONNECT actor authorization and hostname/CIDR destination rules with default deny | S+K (substrate) | Documented as shipped in v1.6.0 (#3237 , #3318 ). Not marked Tested: I have not published a Substrate egress run on v1.6.0. |
Do not assume a standalone-only feature exists on the controller. Dollar budgets are the example in the next section.
Identity, spend and egress: what the category needs, what is shipped
This is not a re-score of the choose-page criteria. It is what the category needs, and what is in a release.
Identity. Policy has to bind to a workload identity, not only a header the agent can set. I ran that path on OSS standalone with SPIFFE: SPIFFE identity for AI agents . The agent should not hold the provider key; the gateway attaches it after the identity check: secretless AI agents (OSS S). MCP’s shipped auth still authenticates the employee more than the unattended agent (MCP agent identity ). The agent identity series is the hub for that gap.
When the agent acts for a person, the next hop needs an exchange, not a forwarded bearer token. The MCP authorization spec (2026-07-28) is explicit: clients MUST NOT send tokens to the MCP server other than ones issued by that server’s authorization server, and servers MUST NOT accept or transit any other tokens. Yu cites the same MUST NOT. I am quoting the spec, not that post’s architecture advice.
For when to use SPIFFE and when to use OAuth, see SPIFFE vs OAuth for AI agents . The exchange itself, run on agentgateway, is in agentgateway token exchange .
Spend. I re-checked the live budget docs on 2026-10-05. They are labelled 1.6 (latest). The standalone per-key page
still documents budgets in USD or Tokens on an API key. The Kubernetes budget-limits page
still describes token-bucket rate limits (unit: Tokens) that reject with 429. The v1.6.0 release notes
refer to USD API-key budgets in standalone. I did not find an open-source Kubernetes per-key dollar budget in those 1.6 docs. So the mode rule still holds in OSS as of 1.6 latest: per-key dollar budgets are standalone-only; Kubernetes OSS budgets are token-based.
The working standalone demo is per-key LLM budgets (OSS S, v1.5.0). Kubernetes dollar budgets in capping LLM spend ran on Solo Enterprise for agentgateway. That page is not OSS evidence.
Egress. In released v1.5.0, substrateEgress only adds actor metadata. The source at tag v1.5.0 still has the stub (// TODO: implement egress policy in Substrate in egress.rs L57
). Actor-identity authorization at CONNECT (PR #3237
) and hostname/CIDR destination rules with deny-when-nothing-matches (PR #3318
, merged 2026-09-09) first appeared in the v1.6.0 prereleases. They shipped in stable v1.6.0 on 2026-10-02 (prerelease: false). The v1.6.0 notes list #3237, #3318, and #3677. The code at tag v1.6.0 egress.rs
is the shipped file.
I am not claiming I re-ran Substrate egress on v1.6.0. The grade is: shipped in v1.6.0, linked from the release. The standalone CONNECT egress proxy docs are a separate, documented S control.
When I write about bypasses, I am writing about the pattern. Egress control used a Python CONNECT stand-in, not agentgateway. It is not agentgateway evidence.
How this differs from “how to choose” and from an agentic mesh
This page is the category definition: what an AI or agent gateway is, how the four labels differ, and what open-source agentgateway covers in S and K.
How to choose an AI agent gateway is the scorecard. Seven controls, how agentgateway meets each in both modes, and PoC tests. Start there when you are writing the ADR. Do not treat this page as that matrix.
What is an agentic mesh is the architecture layer above the hop: workload identity plus an AI-aware data plane. The mesh page is not a second definition of the gateway.
How agentgateway compares is the product comparison. That title is the job of that page. It is not a “when each” handoff.
The agent platform gateway control plane wires the four controls onto one path. What is a cloud-native agent harness covers the runtime the agent lives in, on the far side of the gateway.
Frequently asked questions
What is an AI agent gateway? An AI agent gateway is a reverse proxy at the agent boundary. It governs the traffic agents send to language models, to tools (often over the Model Context Protocol), to other agents (A2A) and to ordinary HTTP or gRPC services, and it applies identity, authorization, budgets, audit and egress on that path. The same product may also be sold as an LLM gateway or an MCP gateway; the useful questions are which traffic it handles and which decisions it is built to make.
How is an AI agent gateway different from an API gateway, an LLM gateway and an MCP gateway? An API gateway governs ordinary service HTTP or gRPC. An LLM (model) gateway governs inference: routing, fallbacks, token or dollar cost, prompt controls. An MCP gateway governs agent-to-tool traffic: discovery, per-tool authorization, sessions and audit. An AI or agent gateway is the broader (or unified) label for governing several of those traffic types together. Labels overlap on real products, so judge by traffic and decisions, not by the badge.
What does agentgateway cover in standalone and Kubernetes modes? Open-source agentgateway is one data plane for HTTP, gRPC, LLM inference, MCP and A2A. Standalone mode is a single binary with YAML, suitable for a laptop, VM or edge box, and the 1.6 (latest) standalone docs still document per-API-key budgets in dollars or tokens, as they did in OSS v1.5.0. Kubernetes mode implements Gateway API with a controller; its open-source budgets are token-based. This page defines the category; the choose guide scores the controls.
Methodology and disclosure
Last checked 2026-10-05. agentgateway docs labelled 1.6 (latest). Releases through v1.6.0 (published 2026-10-02). Standalone and Kubernetes About pages, both budget trees, the Messages and MCP authz pages, the standalone egress-proxy page, and the v1.5.0 / v1.6.0 egress.rs files were fetched live. Peer taxonomy pages: Aembit (Kaplan, September 2026), Ascending (Yu, 2026-09-16), and the Blueprint Alliance press release (Okta, 2026-09-22) plus Okta’s Agent Gateway help and runtime-governance pages (industry framing only). MCP authorization 2026-07-28 for the MUST NOT lines.
Grades are Documented unless a linked demo on this site proves the path, in which case Tested. I have not marked Substrate egress Tested on v1.6.0. I have not invented adoption shares, latencies, or a “most popular” rank.
Solo Enterprise pages cited for dollar budgets on Kubernetes are labelled Solo Enterprise and are not OSS evidence.
Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything above is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project’s own docs or release notes as of 2026-10-05.