Multi-Tenant MCP Federation with agentgateway

Three companies authenticate against three different identity providers, connect to the same three MCP URLs, and get three completely different products. Acme sees 25 tools, Globex sees 13, Initech sees 6. And the six MCP servers behind the gateway contain zero authentication, authorization, quota, or billing code. Every one of those properties is layered in front of them, declaratively, in version control.

The whole thing is in themsquared/agentgateway-federate-mcp-example: a k3d cluster, agentgateway, one Keycloak with three realms, six stub MCP servers, and scripts that show every claim in this post actually working. This is what running MCP servers as a governed, multi-tenant product looks like, and it’s the pattern I’d argue for over the thing I keep seeing teams build instead.

Why not one MCP endpoint with every tool?

The plan I hear most often is one federated endpoint with every MCP server in the company behind it. One URL, 100+ tools, every agent gets the catalog. The problem is measurable: Anthropic’s internal MCP evals scored Claude Opus 4 at 49% with full tool catalogs loaded in context and 74% with on-demand discovery, and they saw real catalogs burn about 77K tokens before the agent did any work. More tools in one namespace makes the agent measurably worse at picking the right one, and the humans reading the list do no better.

The less measurable problem is that a flat namespace has no owner and no boundaries. There is nothing to attach an authorization policy to, nothing to meter against a contract, and no way to give two customers different views of the same platform. Christian Posta has written about how agents break assumptions we carried over from microservices; this is one more of them. The unit that matters isn’t the server you happen to have deployed. It’s the business domain.

So this demo federates six servers into three domains, and every downstream capability (entitlements, quotas, chargeback) hangs off those domain boundaries.

   Acme ─┐                                     ┌─ /mcp/billing    → payments, invoicing
 Globex ─┼─→ JWT ─→  agentgateway  ─→ federation ├─ /mcp/analytics  → reporting, telemetry
Initech ─┘           authn · authz            └─ /mcp/support    → tickets, crm
                     quota · metering

What each company gets:

	Acme (enterprise)	Globex (standard)	Initech (trial)
`/mcp/billing`	8 tools	4 read-only	0
`/mcp/analytics`	8	9 (+ data export add-on)	3 read-only
`/mcp/support`	9	0	3 tickets only
Quota	600/min	60/min	20/min

Look at the analytics row. Globex, the standard-tier customer, has more analytics tools than the enterprise account, because they bought a data export add-on. Entitlements follow commercial agreements, not a tier ladder, and the gateway expresses that directly. That’s the business case for this whole architecture in one table cell: the gateway config is the product catalog.

How the federation works

Each domain is one AgentgatewayBackend that fans a single MCP endpoint across several real servers. A client connects once and sees the union of the targets’ tools:

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayBackend
metadata:
  name: mcp-billing
  namespace: mcp-federation
spec:
  mcp:
    failureMode: FailOpen
    prefixMode: Always
    targets:
    - name: payments
      static:
        host: mcp-payments.mcp-federation.svc.cluster.local
        port: 80
        path: /mcp
        protocol: StreamableHTTP
    - name: invoicing
      static:
        host: mcp-invoicing.mcp-federation.svc.cluster.local
        port: 80
        path: /mcp
        protocol: StreamableHTTP

Two settings here matter more than they look:

prefixMode: Always exposes federated tools as <target>_<tool>, so payments_create_charge and invoicing_void_invoice are stable, predictable names. The authorization rules depend on that.
Static targets, not selectors. A selector: target derives its tool prefix from the discovered Service name (mcp-payments-8080_get_payment), while a static target keeps the name: you declared. When policy matches on tool names, predictable naming is worth the extra three lines.

failureMode: FailOpen keeps the federation serving healthy targets when one is down; the default, FailClosed, fails the whole session.

Each company has its own Keycloak realm, which stands in for each customer bringing their own IdP. One policy lists all three issuers; the gateway reads iss from the presented token, picks the matching provider, and verifies against that provider’s JWKS:

spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: mcp-federation-gateway
  traffic:
    jwtAuthentication:
      mode: Strict
      providers:
      - issuer: http://keycloak.mcp-federation.svc.cluster.local:8180/realms/acme
        audiences: ["mcp-federation"]
        jwks:
          remote:
            url: http://keycloak.mcp-federation.svc.cluster.local:8180/realms/acme/protocol/openid-connect/certs
            cacheDuration: 5m
      # ... globex and initech providers, same shape

mode: Strict closes anonymous access entirely, and audiences pins tokens to this gateway, so a token Acme minted for some other relying party is rejected even with a valid signature. Each token carries company and tier claims, and those two values drive everything that follows. Onboarding a fourth partner is four lines in this file plus an authorization rule; the repo’s scripts/add-partner.sh does it in one command.

How per-customer tool entitlements work

Authorization is CEL expressions on the backend, OR’d together, denied by default:

  backend:
    mcp:
      authorization:
        action: Allow
        policy:
          matchExpressions:
          - 'jwt.company == "acme" && mcp.tool.target == "payments" && mcp.tool.name in ["get_payment", "list_payment_methods", "create_charge", "refund_payment"]'
          - 'jwt.company == "globex" && mcp.tool.target == "payments" && mcp.tool.name in ["get_payment", "list_payment_methods"]'

Initech has no billing entitlement, and its absence from this file is the entitlement. Initech authenticates successfully, reaches /mcp/billing, and sees an empty tool list. You grant access by adding a line, never by remembering to deny one.

Two details I verified against the running gateway rather than assuming, because both are easy to get wrong:

mcp.tool.name is the origin tool name (get_payment), not the federated name the client sees (payments_get_payment). The prefix is presentation; policy matches the real name.
mcp.tool.target is required for correctness, not just tidiness. Without it, a rule allowing get_invoice also allows a same-named tool on any other server in the federation.

Enforcement covers both directions. tools/list is filtered to what the caller may use, and a tools/call for anything else returns Unknown tool, so a caller can’t even confirm that an unentitled tool exists. The repo makes this visible in one command:

scripts/mcp.py matrix

That prints every tool, every company, side by side. My favorite detail: payments_void_transaction and invoicing_void_invoice are implemented, deployed, and running, and no company can reach them. The gateway is the only thing between “the capability exists” and “someone can invoke it”.

Quotas that hold across gateway replicas

Each company gets its own rate limit keyed on the same jwt.company claim the entitlements use:

apiVersion: ratelimit.solo.io/v1alpha1
kind: RateLimitConfig
metadata:
  name: company-quotas
  namespace: mcp-federation
spec:
  raw:
    descriptors:
    - key: company
      value: acme
      rateLimit:
        requestsPerUnit: 600      # enterprise tier
        unit: MINUTE
    - key: company
      value: initech
      rateLimit:
        requestsPerUnit: 20       # trial tier
        unit: MINUTE
    # Catch-all: any company WITHOUT an explicit row gets its own counter
    - key: company
      rateLimit:
        requestsPerUnit: 30
        unit: MINUTE
    rateLimits:
    - actions:
      - cel:
          expression: 'jwt.company'
          key: company

The counters live in the Redis that ships with the enterprise install, so a limit holds across every gateway replica. 600 per minute means 600 total, not 600 per replica, which is the difference between a quota you can put in a contract and one that silently multiplies with your deployment size.

The catch-all row is the part I’d steal for any multi-tenant config: a partner added tomorrow is rate-limited from their first request, under their own counter, without anyone editing this file. The safe default is automatic and an explicit row only ever grants more.

Chargeback from gateway metrics

agentgateway already counts MCP traffic by server, tool, and route. What the metrics lack by default is who called. One policy lifts claims off the validated JWT onto the request metrics:

  frontend:
    metrics:
      attributes:
        add:
        - name: company
          expression: jwt.company
        - name: tier
          expression: jwt.tier

After that, chargeback is a single PromQL query:

sum by (company, server, resource) (agentgateway_mcp_requests_total{method="tools/call"})

scripts/chargeback.py runs it, prices it against a rate card in scripts/pricing.json, and reports allowed versus denied versus throttled per company. --csv emits the same data for a billing pipeline. One cardinality warning from the manifest comments worth repeating: meter on the billable entity (company, tier), never on jwt.sub, or your time series count multiplies by your user count. I covered the LLM-spend version of this same pattern in Capping LLM Spend at the AI Gateway.

Run it yourself

Prerequisites: kubectl, helm, python3, k3d (or an existing cluster), and a Solo agentgateway license key. Validated against Solo Enterprise agentgateway v2026.8.0, Gateway API v1.5.0, Keycloak 26.0, and MCP protocol 2025-06-18.

git clone https://github.com/themsquared/agentgateway-federate-mcp-example
cd agentgateway-federate-mcp-example
cp .env.example .env        # add your AGENTGATEWAY_LICENSE_KEY
./setup.sh                  # creates a k3d cluster and installs everything

In a second terminal:

./port-forward.sh           # gateway :8080, Keycloak :8180, Prometheus :9090, UI :9080

Then either run the guided walkthrough (./demo.sh) or poke at it directly:

scripts/mcp.py list acme billing     # what one company sees
scripts/mcp.py token globex          # a token and its claims
scripts/mcp.py quota initech 25      # watch the trial quota engage
scripts/chargeback.py --by-tool      # usage and cost by customer

The six MCP servers are one stdlib-only Python file on a stock python:3.12-alpine image, with tools defined in ConfigMap JSON. No image builds, no registry. Edit the JSON, re-apply, done.

Gotchas

Things that cost me time, so they don’t cost you any:

Keycloak’s issuer must be pinned. KC_HOSTNAME is set to the in-cluster service URL so tokens minted through kubectl port-forward still carry the in-cluster iss. Without it, a token fetched from your laptop claims iss=http://localhost:8180 and the gateway rejects it.
Policies propagate through xDS. Give it a few seconds after kubectl apply before testing, or a just-applied entitlement looks broken when it’s just not there yet.
Namespace discovery can strand your Gateway. If agentgateway was installed with discoveryNamespaceSelectors (common on a shared demo cluster), the controller ignores non-matching namespaces and the Gateway sits at Waiting for controller with no obvious cause. setup.sh detects this and labels the namespace automatically.
Quota units are HTTP requests, not tool calls. One MCP session spends several (initialize, tools/list, one per call), so trial-sized quotas trip faster than the raw number suggests.
Denied calls still count in agentgateway_mcp_requests_total, which has no status label. chargeback.py cross-references agentgateway_requests_total to separate allowed from denied and throttled; a production rate card should bill on successes only.

Where this lands

The pattern is simple to state: federate MCP servers by business domain, authenticate every caller against their own IdP, express entitlements as CEL in git, and let the same JWT claim drive authorization, quotas, and chargeback. The six servers never learn any of it, which means the next six servers won’t either. That’s the point of putting an AI gateway in front of MCP instead of teaching every server about every customer.

The repo is themsquared/agentgateway-federate-mcp-example, including a WALKTHROUGH that builds it one layer at a time, an ONBOARDING guide for adding a partner with their own IdP, and a PRODUCTION doc mapping the POC to a real estate (50 IdPs, opaque tokens, per-user quotas). Next up, I want to wire agent-facing identity into the same federation: token exchange so the gateway swaps a caller’s JWT for scoped upstream credentials, which is where the agentic mesh story gets interesting.