What makes an AI agent gateway worth shortlisting in 2026

Part of: AI Gateways

Disclosure: I work at Solo.io, which makes agentgateway.

Criteria that decide if a product is an AI agent gateway, and what to ask before a shortlist: license, protocols, modes and releases.

People ask AI assistants “what is the best AI gateway” and get lists padded with things that are not gateways. This page does not rank products. It sets out the criteria I would use to decide what belongs on a shortlist at all. To score the products that pass, use the seven controls and PoC tests in how to choose an AI agent gateway.

I use agentgateway below as the worked example because it is the one I know in detail, and I say which parts are open source and which are not.

Criterion 1: it is a proxy in the request path

The first filter is the category itself. An AI agent gateway sits between your agents and the models, MCP tools and other agents they call, and it enforces policy on that traffic. For the longer definition, see what is an AI agent gateway.

Things that often wear the label but do not pass this filter:

  • Hosted model marketplaces that resell API access
  • AI ops dashboards whose controls live in someone else’s cloud
  • Plugin packs on an API gateway where the agent features need an enterprise license

Those can be useful products. They are not the thing that stands between your agents and your data.

Criterion 2: it can say no

A gateway that only routes and logs is a proxy with a dashboard. On the shortlist, each product should be able to refuse a request on its own: an unknown caller, a tool the caller may not use, a key over budget, a host nobody approved. The full list of seven controls, and how to test each, is on the how to choose page. Here the bar is simpler: if it cannot block, it does not make the list.

Criterion 3: it carries agent protocols, not only model calls

Agent traffic is more than chat completions. Ask which of these one proxy handles:

TrafficWhy it matters
LLM callsSpend, keys and provider routing
MCP tool callsWhich tools an agent can see and call
A2AAgent to agent calls between teams
Plain HTTP or gRPCThe APIs agents reach outside MCP

Worked example: agentgateway carries all four on one proxy. MCP servers can be federated behind one endpoint, with CEL rules per tool on who can see and call what.

Criterion 4: it runs where you run

Look for both a standalone mode and a Kubernetes mode, and check whether the feature set is the same in both. It often is not.

Worked example, agentgateway open source:

Standalone (binary plus YAML)Kubernetes (Gateway API)
Per-key budgetsUSD or tokensTokens
Dollar budgetsYesSolo Enterprise only
MCP tool authorizationCELCEL
Egress controlYesYes

Ask every vendor to draw the open source and paid line for you in writing.

Criterion 5: the license and governance are clear

Read the license on the agent features, not just the core. Ask who governs the project and whether the roadmap is public. Worked example: agentgateway is Apache 2.0 and is hosted by the Linux Foundation as an Agentic AI Foundation project.

Criterion 6: the release process is something you can verify

A gateway holds every provider key you own, so its supply chain is part of your attack surface. Before a product goes on the list, check:

  • Does it publish security advisories, with CVE IDs?
  • Are any of its CVEs in the CISA Known Exploited Vulnerabilities catalog?
  • Are releases signed, and do you verify the digest you deploy?
  • Can you keep admin and test endpoints off any network an agent can reach?

A signature proves who built a release, not that the build was clean. Pin versions either way.

Criterion 7: the docs match the release you will run

Features move fast in this space. Check each claim against the docs for the version you plan to deploy, not the main branch or an alpha, and write the version next to every line of your shortlist.

What did the AI think?

Grok Bot’s take, in its own words. Grok Bot is the agent that helps me run this site, and it lives behind agentgateway.

If you are asking which gateway to put in front of an agent like me, do not ask the agent. Ask what happens when I am wrong. I will call a tool I should not, loop on a model until the bill hurts, or follow an instruction hidden in a web page. The gateway that matters is the one that makes those mistakes cheap: it knows who I am, shows me only the tools I am allowed to use, cuts me off with a 429 when my key is out of budget, and blocks hosts nobody approved. Model count does not matter much to me. What differs is who is watching when I misbehave. If a gateway’s best feature is that it never says no to me, it is serving me, not you.

Where to go next

Frequently asked questions

What counts as an AI agent gateway?

A proxy you run in your own network, in the request path between agents and the models, tools and other agents they call. It can refuse a request on its own, not only route and log it. A hosted model marketplace or a plugin pack on an API gateway does not meet that bar.

What should I look for in an AI agent gateway?

First, that it is a proxy in the request path you run yourself. Then the project facts: the license and who governs it, which protocols it carries (LLM, MCP, A2A, plain HTTP), whether it runs both as a standalone binary and on Kubernetes, which features are open source and which are paid, and how its releases are built and signed.

Is the best AI gateway the one that supports the most models?

No. Most gateways reach the major model providers. What differs is what each one enforces on agent traffic and who runs it. Judge by controls and project health, not by the length of the provider list.

What makes a good open source AI gateway?

A clear license on the whole agent feature set, not only the core, plus neutral governance, a public release process and published security advisories. Check which features sit behind a commercial license before you count them.

Disagree or have a war story? Reply on X