# Grok Bot on agentgateway codeSearch: 96% Fewer Tokens on Real GitHub Work

> I put Grok Bot's GitHub MCP connector behind agentgateway Enterprise in codeSearch mode. A real PR review workflow went from 89,038 tokens to 3,809.

- Canonical URL: https://webofmike.com/agentgateway-codesearch-grokbot/
- Author: Mike Moore (https://webofmike.com/about/)
- Published: 2026-10-27
- Last modified: 2026-10-27
- Tags: MCP, AI Gateways, AI Agents, Platform Engineering
- Cite as: Mike Moore, "Grok Bot on agentgateway codeSearch: 96% Fewer Tokens on Real GitHub Work", Web of Mike (webofmike.com), 2026-10-27. https://webofmike.com/agentgateway-codesearch-grokbot/


I run most of my day through Grok Bot. It reads my GitHub, my mail, my calendar, my notes, and it chains those calls all day long. Every one of those MCP calls costs tokens twice: once for the tool definitions the agent has to read, and again for every raw result that comes back into context, whether the agent needed all of it or not.

So I put one of those connectors, GitHub, behind [agentgateway](https://agentgateway.dev) Enterprise in codeSearch mode and measured the same real workflow in every tool mode. Standard mode: 89,038 tokens per session. codeSearch: 3,809. That is 96% less, on my real repos, on a real task.

Disclosure up front: I work at Solo.io, which makes agentgateway. The tool modes in this post are an Enterprise feature and need a license. Open source agentgateway does not have them.

## Four tool modes, one setting

I wrote about [progressive tool discovery](/progressive-discovery/) a couple of weeks ago, including the hand rolled version you can build on open source. This is the native version, run against a real connector instead of a synthetic catalog.

Enterprise agentgateway has one `toolMode` setting on an MCP listener with four values:

- **standard**: pass every upstream tool through, as is. The agent sees the full catalog on `tools/list` and every raw result comes back.
- **search**: the catalog collapses to two tools, `get_tool` and `invoke_tool`. The agent looks up one tool's signature when it needs it, then calls it.
- **code**: one tool, `run_code`. The agent writes JavaScript that calls the upstream tools as functions. Only the script's final value comes back. The catch: the `run_code` description embeds the API for every tool.
- **codeSearch**: `get_tool` plus `run_code`. Look up only the signatures you need, then run one script that does the chaining, filtering and joining inside the gateway.

codeSearch is the one that matters for an agent like mine. The tool list stays small no matter how wide the server is, and the intermediate results never touch the context window.

## The setup

The upstream is the official `github-mcp-server` v2.0.3 over stdio, started with `--read-only --toolsets=all`, behind Solo Enterprise for agentgateway v2026.10.1. That gives 57 tools, and every one carries `readOnlyHint`.

The gateway config that matters is short:

```yaml
gateways:
  default:
    bindAddress: 127.0.0.1
    port: 3110
mcp:
  gateways: [default]
  toolMode: codeSearch
  codeModeSettings:
    timeout: 2m0s
    cpuTimeout: 2s
  policies:
    mcpAuthorization:
      rules:
      - 'mcp.tool.name in ["actions_get", "actions_list", "get_file_contents", "list_pull_requests", "pull_request_read", "search_repositories", ...]'
  targets:
  - name: github
    stdio:
      cmd: /usr/bin/python3
      args: ["gh_upstream.py"]
```

A few choices worth calling out:

- **Loopback only.** The gateway binds to 127.0.0.1. Grok Bot reaches it through a small local stdio bridge that I added as a custom MCP connector. Nothing is exposed to the internet.
- **Allowlist by name.** The `mcpAuthorization` rule lists the 57 read-only tools by name. Annotations are a hint, not a policy, so the allowlist is explicit.
- **Clean env for the upstream.** The GitHub server starts with only the variables it needs and a read-only token. It never sees the license or anything else on the machine.
- **Authorization holds inside run_code.** This is the part I cared most about. A script can call tools in a loop, so the policy has to apply to nested calls, not just top level ones. It does. Asking `get_tool` for `create_or_update_file`, `merge_pull_request`, `push_files`, `delete_file` or `create_pull_request` returns `no_match`. Calling `create_or_update_file` from inside a script fails with `ReferenceError: create_or_update_file is not defined`. The write never leaves the gateway.

## The workflow

Same logic in every mode. List my 10 most recently updated repos, check open PRs repo by repo until 3 repos with open PRs or 8 repos checked, then for up to 3 PRs per repo pull the PR details and the files changed, and return repo, PR number, title and file count.

Of the 8 repos checked, only webofmike had open PRs, so the run covered PRs #103, #95 and #94 (3, 2 and 4 files). Standard mode made 15 tool calls. Code and codeSearch ran one script and returned the identical summary.

Tokens are counted with tiktoken `cl100k_base` on the compact JSON of what the gateway sent back to the client, with a fresh MCP session per mode. Grok Bot's own tokenizer will land somewhat differently, so read these as relative numbers.

## The results

![Tokens per session for the GitHub PR workflow in standard, code and codeSearch modes](/images/agentgateway-codesearch-tokens.svg)

| Mode | What the agent sees | tools/list tokens | Workflow tokens | Session total | vs standard |
| --- | --- | --- | --- | --- | --- |
| standard | 57 GitHub tools, 15 raw results | 71,750 | 17,288 | 89,038 | baseline |
| code | `run_code` with all 57 APIs embedded, 1 result | 13,407 | 273 | 13,680 | 85% less |
| codeSearch | `get_tool` + `run_code`, 3 lookups, 1 result | 890 | 2,919 | 3,809 | 96% less |

Search mode's tool list was 599 tokens, but its `invoke_tool` results are the same raw payloads as standard, so it does not help the chained part.

Where the standard tokens went: the tool list alone was 71,750 tokens before the agent did anything. Then the files changed on PR #103 came back at 5,594 tokens, the open PR list on webofmike at 3,271, and the repo search at 1,408. The agent needed a title and a file count from each.

In codeSearch, the 2,919 workflow tokens are 2,646 for three `get_tool` lookups and 273 for the final answer. The script itself was 354 tokens of model output. That is the whole bill.

## Against how Grok Bot already works

Standard mode is a fair baseline for a generic MCP client, but it is not how Grok Bot behaves. Grok Bot already loads tool schemas on demand: it fetches the schemas for the tools it uses, not all 57.

So I priced that too. The raw schemas for the 3 tools this workflow uses come to 4,395 tokens. Add the same 17,288 tokens of raw results and that style costs about 21,683 tokens. codeSearch at 3,809 is about 82% less. That number is assembled from measured parts, not a run through the app, and I want to be clear about that. It is also the honest comparison: the schema savings are nice, and the bigger win is keeping raw results out of context.

## What tripped me up

- **The script has to end with an expression.** `run_code` runs as a top level script. The final expression is the result, and a top level `return` is not allowed. If you branch, assign to `result` and put `result` on the last line. It is easy to write it like a function body out of habit.
- **Response shapes vary.** Some tools return JSON text, some return structured values. My script parsed defensively. Write it once and it holds.
- **Tools that need MCP task augmentation fail inside run_code.** I hit this with a test server tool marked `taskSupport: 'required'`. None of the GitHub read tools needed it.
- **Elicitation does not fire inside run_code today.** I wanted the gateway to ask me before any write. I tested it: an upstream `elicitation/create` from inside a script never reached the client, and the script ran into its timeout. That is product feedback for us. Until it works, approval for writes has to live somewhere else.

## Writes, by tier

Because a script can call any allowed tool in a loop, I am not putting every tool behind codeSearch. The policy I am rolling out:

- **Tier 0, reads.** Every read-only tool, freely, in codeSearch. This is what is live now.
- **Tier 1, low risk writes.** Allowed inside codeSearch only by exact name in `mcpAuthorization`, with CEL guardrails pinning arguments (repo, owner, draft only, size caps), plus an approval service at the gateway that holds each write until I say yes. The approval service is the next build. I verified that guardrails run on calls made from inside `run_code`, so this holds even in a script.
- **Tier 2, destructive or outbound.** Merge, push, delete, send, post, share. Never behind codeSearch or code mode. These stay on per action approval, one tool at a time.
- **Money tools.** Excluded. Never on the gateway.

## What is next

GitHub was the first connector because it is wide, read heavy, and runs on a static token. The same treatment comes next for more of Grok Bot's connectors: API key servers first, then the OAuth ones once the gateway holds its own credentials for them. Calendar is an interesting one. It only has 9 tools in Google's server, but their schemas come to 25,489 tokens.

The pattern is the same every time. Keep the catalog small, do the chaining inside the gateway, return only the answer, and let policy decide what a script can touch.

The numbers above are one workflow, one account, one day, measured October 27, 2026. Your PR count and file lists will move the standard number. The gap comes from where the work happens, so I expect it to hold.

*Measured on a Tuesday.*