# agentgateway Guardrails How-To

> Apply AI gateway guardrails with agentgateway: regex PII mask/reject, webhooks, Bedrock, Model Armor. S+K. Streaming and fail-open notes. Solo.io disclosure.

- Canonical URL: https://webofmike.com/agentgateway-guardrails/
- Author: Mike Moore (https://webofmike.com/about/)
- Published: 2026-10-21
- Last modified: 2026-10-21
- Tags: AI Gateways, AI Agents, Security, Platform Engineering, Kubernetes, Generative AI
- Cite as: Mike Moore, "agentgateway Guardrails How-To", Web of Mike (webofmike.com), 2026-10-21. https://webofmike.com/agentgateway-guardrails/


Agents and apps already send prompts through a gateway. Content safety belongs on that hop: inspect the request before it reaches a model, and inspect the response before it reaches a client. This page is the graded how-to for open-source **agentgateway** prompt guards. It walks regex builtins and custom patterns (mask, reject, audit), custom webhooks, AWS Bedrock Guardrails, and Google Model Armor in **standalone and Kubernetes** modes, with the streaming, `failureMode`, and guard-scope edges called out. It is not the [Solo Enterprise WAF that reads the prompt](/waf-for-llm-and-mcp-traffic/) with Coraza and the OWASP Core Rule Set.

*Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything below is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project's own docs or release notes as of 2026-10-05.*

Mode key: **S** is standalone YAML, **K** is Kubernetes `AgentgatewayPolicy` `promptGuard`, **S+K** is both. Field names differ. Grades are **Tested** only where Marketing ran the open-source standalone binary **v1.6.0** on 2026-10-05; everything else is **Documented**.

<!-- TODO once live (404 as of 2026-10-05): /what-is-an-ai-agent-gateway/ (category definition; guardrails are one shipped capability) -->

## What AI gateway guardrails are

AI gateway guardrails are content-safety checks an AI or agent gateway runs on LLM requests and responses, and, when you configure it, on tool-related fields. Each check can allow the content through, reject it, mask matched spans, or audit the match without changing traffic. The gateway is the enforcement point, so every client behind it inherits the same policy.

```text
client
  |
  v
agentgateway
  1. regex (fast, deterministic)
  2. optional external guard (Bedrock, Model Armor, OpenAI Moderation, Azure Content Safety)
  3. optional custom webhook
  |
  v
LLM
  |
  v
agentgateway  (response guards, same layers)
  |
  v
client
```

Guards run only on LLM conversation routes: `completions`, `messages`, `responses`, and `generateContent` (**S+K**, Documented). `realtime` runs only when streaming is enabled, and mask still does not rewrite WebSocket frames. Embeddings, rerank, token-count, `models`, `passthrough`, and `detect` skip guards. Authorization still sits in front; two [CEL authorization gotchas](/agentgateway-cel-authorization-gotchas/) can fail open or 403 every call before a guard runs.

## agentgateway's guardrail layers

The [standalone overview](https://agentgateway.dev/docs/standalone/latest/llm/prompt-guards/overview/) and the [Kubernetes overview](https://agentgateway.dev/docs/kubernetes/latest/llm/guardrails/overview/) describe the same three layers (**S+K**, Documented):

| Layer | What it does | Actions | Default action |
| --- | --- | --- | --- |
| Regex | Built-in PII patterns and custom expressions | mask / reject / audit (**K:** Mask / Reject / Audit) | mask |
| External moderation | OpenAI Moderation, AWS Bedrock Guardrails, Google Model Armor, Azure Content Safety | reject / audit | reject |
| Custom webhook | Your detector, via `POST /request` and `POST /response` | reject / audit | reject |

Mask is a regex capability. External guards and webhooks reject or audit. A webhook can still return its own mask verdict (the sample server turns the word `mask` into `****`).

Default rejection is HTTP **403** with the body `The request was rejected due to inappropriate content` (**S+K**, Documented). Standalone customizes that with `rejection.status`, `rejection.body`, and `rejection.headers`. Kubernetes uses `response.message` and `response.statusCode`. Webhooks ignore those fields and set their own status and body.

On standalone, `llm.policies.guardrails` is a shared baseline for every model, and `llm.models[].guardrails` adds per-model guards. The two lists merge. Kubernetes attaches `promptGuard` on an `AgentgatewayPolicy` that targets a route.

## Regex filters: builtins, custom patterns, mask vs reject

[Standalone regex](https://agentgateway.dev/docs/standalone/latest/llm/prompt-guards/regex/) and [Kubernetes regex](https://agentgateway.dev/docs/kubernetes/latest/llm/guardrails/regex/) ship the same builtins with different names (**S** / **K**, Documented):

| Standalone (`builtin`) | Kubernetes (`builtins`) | Note |
| --- | --- | --- |
| `email` | `Email` | Email addresses |
| `phoneNumber` | `PhoneNumber` | Phone numbers |
| `ssn` | `Ssn` | Social Security numbers |
| `creditCard` | `CreditCard` | Luhn checksum required. A number that fails Luhn is not masked, rejected, or audited |
| `caSin` | `CaSin` | Canadian Social Insurance Numbers |

Custom patterns are first-class. Standalone uses `rules` with `pattern`. Kubernetes uses `regex.matches`. The Kubernetes docs include example sets for instruction-override phrases, credentials, and jailbreak framing. Those examples are **Documented**, not a production pack I tested.

### Request reject, both modes

Marketing's standalone harness (P2, **Tested**) rejects an email with a custom 400 body (`code: pii_detected`) and rejects the keywords `SSN` / `Social Security` with `code: content_policy_violation`. A clean `"Hello"` reaches the mock upstream at HTTP 200.

Standalone (**S**, Tested on v1.6.0). A second request rule used `pattern: SSN` / `Social Security` and `code: content_policy_violation`:

```yaml
guardrails:
  request:
  - regex:
      action: reject
      rules:
      - builtin: email
    rejection:
      status: 400
      headers:
        set:
          content-type: "application/json"
      body: '{"error":{"message":"Request blocked: Contains email address","type":"invalid_request_error","code":"pii_detected"}}'
```

Curl outcomes from that run:

```text
POST /v1/chat/completions  {"messages":[{"role":"user","content":"Hello"}]}
-> HTTP 200   mock assistant reply

POST ...  "Contact me at test@example.com"
-> HTTP 400   {"error":{"code":"pii_detected", ...}}

POST ...  "My SSN is on file"
-> HTTP 400   {"error":{"code":"content_policy_violation", ...}}
```

The Kubernetes twin is the same policy under `AgentgatewayPolicy` (**K**, Documented; P8 was not run):

```yaml
apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: regex-request-reject
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: openai
  backend:
    ai:
      promptGuard:
        request:
        - regex:
            action: Reject
            builtins: [Email]
          response:
            message: "Request blocked: Contains email address"
            statusCode: 400
        - regex:
            action: Reject
            matches: ["SSN", "Social Security"]
          response:
            message: "Request rejected: Content contains sensitive information"
            statusCode: 400
```

### Response mask, both modes

Standalone response mask on builtin `phoneNumber` (**S**, Tested, P3, `stream: false`). The mock returned `919 222 1111`. The assistant content came back with `<PHONE_NUMBER>` and the raw digits were gone. The access log recorded `phase":"response","guard":"regex","action":"mask"`.

```yaml
guardrails:
  response:
  - regex:
      action: mask
      rules:
      - builtin: phoneNumber
```

Kubernetes (**K**, Documented):

```yaml
      promptGuard:
        response:
        - regex:
            action: Mask
            builtins:
            - PhoneNumber
```

Mask on a buffered response is the path that works. Mask on a stream is the F8 hole below.

## Custom webhooks

A webhook is how you bring your own detector. agentgateway calls it synchronously on the request path and, if configured, on the response path.

Standalone ([webhooks](https://agentgateway.dev/docs/standalone/latest/llm/prompt-guards/webhooks/), **S**, Documented; **Tested** in P5): `webhook.target.host` is a `host:port` or an `http(s)` URL. Defaults are `POST /request` and `POST /response`. Optional `headers` is a CEL map (including `:path`). Default timeout is 10 seconds; override with `target.policies.http.requestTimeout` or a named backend. `failureMode` is `failClosed` unless you set `failOpen`.

```yaml
guardrails:
  request:
  - webhook:
      target:
        host: "127.0.0.1:18081"
      failureMode: failClosed
  response:
  - webhook:
      target:
        host: "127.0.0.1:18081"
      failureMode: failClosed
```

Kubernetes (**K**, Documented): `webhook.backendRef` points at a Service. The [webhook guide](https://agentgateway.dev/docs/kubernetes/latest/llm/guardrails/webhook/guardrails/) deploys a sample `gcr.io/solo-public/docs/ai-guardrail-webhook:latest` that rejects `block` and masks `mask` to `****`. Timeout is a second `AgentgatewayPolicy` with `backend.http.requestTimeout` on that Service. Pooling knobs are `GUARDRAIL_WEBHOOK_MAX_CONNECTIONS` and `GUARDRAIL_WEBHOOK_KEEPALIVE_SECONDS` on `AgentgatewayParameters`.

```yaml
      promptGuard:
        request:
        - webhook:
            backendRef:
              kind: Service
              name: ai-guardrail-webhook
              port: 8000
        response:
        - webhook:
            backendRef:
              kind: Service
              name: ai-guardrail-webhook
              port: 8000
```

Guard-level `rejection` / `response.message` is ignored for webhooks. The webhook owns the status and body.

P5 against a local sample server (**S**, Tested): a clean prompt was allowed (HTTP 200). A prompt containing `block` returned HTTP **403** with the webhook body `request blocked by harness webhook`. A request containing `mask` logged `action":"mask"`. A response containing `mask` came back with `****` in the assistant text.

## AWS Bedrock Guardrails and Google Model Armor

Both attachments are **model-agnostic**. They apply to request and/or response, regardless of where the upstream model is hosted (**S+K**, Documented). Marketing had no cloud credentials or pre-created guardrail or template, so P7 is **SKIP, Documented OK**. Nothing below is Tested.

Standalone Bedrock ([docs](https://agentgateway.dev/docs/standalone/latest/llm/prompt-guards/bedrock-guardrails/)) uses `guardrailIdentifier`, `guardrailVersion`, `region`, and `policies.backendAuth.aws`. Kubernetes ([docs](https://agentgateway.dev/docs/kubernetes/latest/llm/guardrails/bedrock-guardrails/)) uses `identifier`, `version`, `region`, and `policies.auth.aws` (`secretRef` or `{}`). Optional `http.requestTimeout` sits under `policies`.

```yaml
# Standalone (S, Documented)
- bedrockGuardrails:
    guardrailIdentifier: <your-guardrail-id>
    guardrailVersion: DRAFT
    region: us-west-2
    policies:
      backendAuth:
        aws: {}
```

```yaml
# Kubernetes (K, Documented)
- bedrockGuardrails:
    identifier: <guardrail-ID>
    version: "DRAFT"
    region: us-west-2
    policies:
      auth:
        aws:
          secretRef:
            name: aws-secret
```

Bedrock is one of two guard types that accept a non-default `scope` (the other is regex). A Social Security number that arrives only in a tool result is invisible unless `scope` lists `toolOutput`.

Standalone Model Armor ([docs](https://agentgateway.dev/docs/standalone/latest/llm/prompt-guards/google-model-armor/)) takes `templateId`, `projectId`, `location`, and `policies.backendAuth.gcp`. Kubernetes ([docs](https://agentgateway.dev/docs/kubernetes/latest/llm/guardrails/google-model-armor/)) uses the same identifiers under `promptGuard`, with `policies.auth.gcp` (for example `type: AccessToken`) and documented kind or GKE workload-identity setup.

```yaml
# Standalone (S, Documented)
- googleModelArmor:
    templateId: <your-template-id>
    projectId: <your-project-id>
    location: us-central1
    policies:
      backendAuth:
        gcp: {}
```

```yaml
# Kubernetes (K, Documented)
- googleModelArmor:
    templateId: <template-ID>
    projectId: <project-ID>
    location: us-central1
    policies:
      auth:
        gcp:
          type: AccessToken
```

OpenAI Moderation is in the same external family. Azure Content Safety is listed on the standalone overview. Both stay **Documented**; this page does not walk them.

## Streaming, failure modes, and scope

### Streaming (F8)

Default: response guards do **not** run when the client sets `"stream": true` (**S+K**, Documented). Enable them with `streaming: Enabled` on standalone (`llm.policies.guardrails.streaming` or per-model) or `promptGuard.streaming: Enabled` on Kubernetes.

**Mask cannot rewrite content that is already on the way to the client.** A matching response `action: mask` (or `Mask`) on a stream passes that content through unmodified. No client error. No guardrail event. The same limit applies to a webhook mask and to Bedrock anonymization on a stream. Request guards do not have this hole: the request is buffered, so mask and reject still apply.

P4a (**S**, Tested, F8 honesty): `streaming` unset (default Disabled), response `action: mask` on `phoneNumber`, client `"stream": true`. SSE HTTP 200. Raw `919 222 1111` in the stream. No `<PHONE_NUMBER>` placeholder. No response-phase `agw.ai.guardrails` action in the access log.

P4b (**S**, Tested): `streaming: Enabled`, response `action: reject` on `phoneNumber`, client `"stream": true`. The SSE response stayed HTTP 200. The stream ended with an in-band error:

```text
data: {"error":{"type":"guardrail_blocked","code":"guardrail_intervention","message":"...pii_detected..."}}
```

Raw phone digits were absent. The log recorded `action":"reject"`. To protect a streamed response, enable streaming and **reject**. Do not rely on mask.

### failureMode (F6)

`failureMode` applies to webhook, OpenAI Moderation, Bedrock Guardrails, Google Model Armor, and Azure Content Safety (**S+K**, Documented). Default `failClosed` / `FailClosed` rejects when the provider is unreachable or returns an error. `failOpen` / `FailOpen` lets the content continue.

A provider error is not a verdict. `action: audit` (or `Audit`) still incurs provider latency and cost, and it does **not** change error handling. An audit guard with the default `failureMode` still rejects when the provider call fails.

### Scope (F7)

A request guard does not read the whole request. Default scope is `systemPrompt` + `messages` (**S**), or `SystemPrompt` + `Messages` (**K**). Tool results pass unless `scope` includes `toolOutput` / `ToolOutput`.

**Scope replaces the default. It does not add to it.** `scope: [toolOutput]` reads tool results and stops reading messages. To cover both, list both values. An empty list is rejected.

Only **regex** and **bedrockGuardrails** accept a non-default scope. Any other guard type fails to load if `scope` is set. The field applies to request guards only. Masking `toolInput` can produce invalid JSON, because chat-completions tool arguments are one opaque string.

```yaml
# Standalone (S, Documented)
request:
- scope: [systemPrompt, messages, toolOutput]
  regex:
    action: mask
    rules:
    - builtin: ssn
```

```yaml
# Kubernetes (K, Documented)
request:
- scope:
  - SystemPrompt
  - Messages
  - ToolOutput
  regex:
    action: Mask
    builtins:
    - Ssn
```

## Proof: Marketing-owned harness (v1.6.0)

Marketing ran the open-source standalone linux-amd64 binary on 2026-10-05 09:19:35 PDT. Release [v1.6.0](https://github.com/agentgateway/agentgateway/releases/tag/v1.6.0). Reported `version: 1.6.0`, `git_revision: ea5608642b9d5c94c5baf5fd9f2f9849b808e963`. Checksum `496ed27e6a96b55a9ad7cabe5c2be7172511f44a3cb113e533d22bf93a7f079d` (`sha_match=true`). Upstream: local mock `POST /v1/chat/completions` on `127.0.0.1:18080`. Listener `http://127.0.0.1:4000`. No provider key, Docker, Kubernetes, or cloud guardrail account.

The harness is Marketing box artifacts (README labelled v1.6.0, test date above). It is not a public clone in this repo. This PR does not publish the binary.

| ID | Case | Grade | Outcome |
| --- | --- | --- | --- |
| P1 | Binary version + sha256 | **Tested (S)** | 1.6.0; sha matches the release |
| P2 | Regex request reject (`email` + `SSN` / `Social Security`); custom 400 bodies; clean Hello | **Tested (S)** | Hello → 200; email → 400 `pii_detected`; SSN keyword → 400 `content_policy_violation` |
| P3 | Regex response mask `phoneNumber`, `stream: false` | **Tested (S)** | Assistant text contains `<PHONE_NUMBER>`; raw `919 222 1111` absent; log `action":"mask"` |
| P4a | `stream: true` + response mask + `streaming` unset | **Tested (S), F8 honesty** | SSE 200; raw phone pass-through; no placeholder; no response-phase guard action |
| P4b | `streaming: Enabled` + response reject + `stream: true` | **Tested (S)** | SSE ends `error.type=guardrail_blocked` / `code=guardrail_intervention`; raw phone absent |
| P5 | Local webhook allow / reject / mask | **Tested (S)** | Clean → 200; `block` → 403 + webhook body; request `mask` → log mask; response `mask` → `****` |
| P6 | Access log lines for guard hits | **Tested (S)** | `agw.ai.guardrails=[...]` on reject, mask, allow, and webhook actions |
| P7 | Bedrock / Model Armor | **Documented** | SKIP; no cloud credentials or pre-created guardrail or template |
| P8 | Kubernetes `AgentgatewayPolicy` twin | **Documented** | SKIP; standalone-only harness |

P6 excerpt, request reject (email):

```text
http.status=400 ... agw.ai.guardrails=[{"phase": "request", "guard": "regex", "action": "allow"}, {"phase": "request", "guard": "regex", "action": "reject"}] error="request rejected by regex guardrail" reason=Guardrail
```

P3 excerpt, response mask: `agw.ai.guardrails=[{"phase": "response", "guard": "regex", "action": "mask"}]`. P4a logged `http.status=200` and did **not** record a response-phase guardrail action. That is the F8 honesty note, not a missed scrape.

## Not the WAF page (and not egress)

[A WAF that reads the prompt](/waf-for-llm-and-mcp-traffic/) is **Solo Enterprise for agentgateway**: Coraza plus the OWASP Core Rule Set, `HeadersAndBody`, a JSON body processor, and custom SecLang on LLM prompts and MCP JSON-RPC. The CRDs are `WAFPolicy` and `EnterpriseAgentgatewayPolicy`. That lab ran on v2026.8.2. It is HTTP and signature inspection, not OSS prompt guards.

This page is the OSS companion: regex, webhook, Bedrock Guardrails, Model Armor, on LLM conversation routes. A CRS 403 is not proof that `promptGuard` fired.

Egress is a different control. A CONNECT hostname allowlist decides *where* an agent may go, not what a prompt contains.

<!-- TODO once live (404 as of 2026-10-05): /agentgateway-egress-proxy/ (destination allowlist, not content safety) -->

<!-- TODO once live (404 as of 2026-10-05): /agent-platform-gateway-control-plane/ (guardrails as one control among others) -->

Per-key dollar and token budgets are not this page; see [per-key LLM budgets](/agentgateway-per-key-llm-budgets/) (OSS standalone) and the labelled Solo Enterprise [cost-controls walkthrough](/llm-cost-controls-ai-gateway/).

The scorecard that lists content safety as one criterion is [how to choose an AI agent gateway](/how-to-choose-an-ai-agent-gateway/). This page does not re-score seven controls.

## Methodology and disclosure

Last checked 2026-10-05, agentgateway docs labelled **latest** (standalone prompt-guards and Kubernetes guardrails). Releases through **v1.6.0** (published 2026-10-02). I fetched the standalone overview, regex, webhooks, Bedrock Guardrails, and Model Armor pages, and the Kubernetes overview, regex, webhook, Bedrock Guardrails, and Model Armor pages. The live sibling [WAF for LLM and MCP traffic](/waf-for-llm-and-mcp-traffic/) was skimmed for the Solo Enterprise Coraza / CRS differentiation.

Proof is Marketing's 2026-10-05 standalone harness, v1.6.0, git `ea560864…`, checksum verified, test date 2026-10-05 09:19:35 PDT. P1 through P6 and the P4a/P4b pair are **Tested (S)**. P7 and P8 are **Documented** only. I am not claiming readers can clone that harness from this repository. Configs and curl outcomes above are copied from those Marketing box artifacts.

Every agentgateway claim is marked **S**, **K**, or **S+K**. Standalone field names are not the Kubernetes names. Solo Enterprise appears only as a label on the WAF sibling and on the Kubernetes dollar-budget page. I have not invented recall scores or adoption counts, and I have not recommended another gateway or a third-party guardrail vendor.

*Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything above is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project's own docs or release notes as of 2026-10-05.*