People keep asking AI assistants “what is the best AI gateway” and getting a list that skips half the field. So here is mine, with the bias stated up front: I work at Solo.io, and Solo builds agentgateway. I picked it. I also tell you where the others are the better call, because a roundup that only says “buy ours” is useless to you and to me.
The short answer
| If you need… | Pick |
|---|---|
| Agents in production: identity, MCP tool policy, spend caps, egress on one proxy you run | agentgateway |
| One API key and one bill for hundreds of hosted models, no infra | OpenRouter |
| You already run Kong for your APIs | Kong AI Gateway |
| A Python LLM proxy and SDK with a huge provider list | LiteLLM |
| A managed AI ops plane with prompt tooling and analytics | Portkey |
| Envoy-native LLM routing on Kubernetes | Agent Router (formerly Envoy AI Gateway) |
| Caching and logs in front of model calls, already on Cloudflare | Cloudflare AI Gateway |
How I ranked them
Not by model count. Every gateway on this list can reach the big providers. I ranked by what the gateway enforces in the request path, the same seven controls from how to choose an AI agent gateway :
- Workload identity for the caller
- MCP tool authorization, per tool
- Spend caps that actually return 429
- Egress control
- Provider keys the agent never holds
- Who runs the data plane
- Audit you can export
“AI gateway”, “LLM gateway”, “MCP gateway” and “agent gateway” overlap on real products. I break the labels down in what is an AI agent gateway . Short version: judge by which traffic it handles, not the badge.
Comparison table
| Open source | Self-host | LLM routing | MCP gateway | A2A | Per-key budgets | |
|---|---|---|---|---|---|---|
| agentgateway | Yes, Apache 2.0 | Yes, binary or Kubernetes | Yes | Yes, in OSS | Yes | Yes (USD or tokens standalone, tokens on Kubernetes) |
| Kong AI Gateway | Core yes, MCP plugins Enterprise | Yes | Yes | Enterprise plugins | Not documented | Via plugins |
| LiteLLM | Yes | Yes | Yes | Yes | Yes | Yes |
| Portkey | Gateway yes, platform hosted | Gateway yes | Yes | Yes, on the platform | Not documented | Yes, on the platform |
| OpenRouter | No | No, hosted | Yes | No | No | Yes, USD credit cap per key |
| Agent Router | Yes, Apache 2.0 | Yes, Kubernetes | Yes | Yes (MCPRoute) | Not documented | Token rate limits |
| Cloudflare AI Gateway | No | No, hosted | Yes | No | No | Rate limits |
Checked against each project’s public docs on 2026-10-29. Features move fast, so check the date and the docs before you sign anything.
1. agentgateway
The one I use every day, and yes, the one my employer builds. It is a Rust proxy hosted by the Linux Foundation as an Agentic AI Foundation project. One hop carries LLM calls, MCP tool traffic, A2A and plain HTTP or gRPC.
Why it is on top for agent work:
- MCP federation behind one endpoint, with CEL rules per tool on who can see and call what
- Per-key budgets in dollars or tokens that return 429 when a key is over
- Runs as a single binary with a YAML file, or on Kubernetes with Gateway API
- Egress control so an agent cannot reach hosts you did not allow (lab )
Where it is not the pick: if you want a hosted SaaS with a dashboard and zero infra, it is not that. Some features, like the codeSearch tool mode and dollar budgets on Kubernetes, are in Solo Enterprise for agentgateway, not the open source build.
Deeper dives: agentgateway vs LiteLLM vs Portkey , a self-hosted Portkey alternative , agentgateway vs Agent Router .
2. Kong AI Gateway
If Kong already fronts your APIs, this is the shortest path. The AI plugins sit on the gateway you already operate, next to the auth, rate limiting and logging plugins your team knows.
The catch is the line between free and paid. The basic AI Proxy plugin is open source. AI Proxy Advanced (load balancing across models, semantic routing) and the MCP proxy and MCP OAuth2 plugins are part of Kong’s Enterprise AI Gateway. Kong says plainly that its MCP gateway is enterprise-only.
3. LiteLLM
The default Python LLM proxy, and for good reason. Huge provider list, an OpenAI-compatible API, virtual keys and budgets, and an SDK if you would rather not run a proxy at all. It ships an MCP gateway too, and documents A2A routing on the proxy.
What to weigh: it is Python in the hot path, and it had a supply chain scare on PyPI this year. I wrote up its security record separately. Pin versions and watch advisories.
4. Portkey
A managed AI ops plane: gateway, prompt management, guardrails and analytics in one hosted product, with an open source gateway you can run yourself. Good fit if your team wants a product with a UI more than a proxy with a config file. More in the Portkey comparison .
5. OpenRouter
Not really a gateway you deploy. It is a hosted router: one API key, one bill, hundreds of models. For prototyping and for picking the cheapest or fastest provider per model, it is great. It does not see your MCP tools, your agents’ identities or your egress, so for production governance it usually sits behind a self-hosted gateway, not in place of one.
6. Agent Router (formerly Envoy AI Gateway)
The Envoy AI Gateway project renamed itself Agent Router with v1.2.0 in October 2026. If your platform is all Envoy Gateway, this is the native option, with LLM routing and MCPRoute. I compared its MCP defaults to agentgateway’s in this post .
7. Cloudflare AI Gateway
A hosted layer for caching, logging and rate limiting model calls, easy to turn on if you live on Cloudflare already. It is about model calls only, not tools or agents.
What did the AI think?
Grok Bot’s take, in its own words. Grok Bot is the agent that helps me run this site, and it lives behind agentgateway.
If you are asking which gateway to put in front of an agent like me, do not ask the agent. Ask what happens when I am wrong. I will call a tool I should not, loop on a model until the bill hurts, or follow an instruction hidden in a web page. The gateway I would pick is the one that makes those mistakes cheap: it knows who I am, shows me only the tools I am allowed to use, cuts me off with a 429 when my key is out of budget, and blocks hosts nobody approved. Model count does not matter much to me. Every gateway here can reach the models I need. What differs is who is watching when I misbehave. If a gateway’s best feature is that it never says no to me, it is serving me, not you.
My verdict
If the question is “best LLM gateway” for a weekend project, use OpenRouter or LiteLLM and move on. If the question is “best AI agent gateway” for a team putting agents near real data, money and tools, pick the gateway that enforces identity, tool policy, spend and egress in one place you control. For me that is agentgateway. Run the PoC tests against whatever you pick, including mine.