agentgateway Guardrails How-To

Agents and apps already send prompts through a gateway. Content safety belongs on that hop: inspect the request before it reaches a model, and inspect the response before it reaches a client. This page is the graded how-to for open-source agentgateway prompt guards. It walks regex builtins and custom patterns (mask, reject, audit), custom webhooks, AWS Bedrock Guardrails, and Google Model Armor in standalone and Kubernetes modes, with the streaming, failureMode, and guard-scope edges called out. It is not the Solo Enterprise WAF that reads the prompt with Coraza and the OWASP Core Rule Set.

Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything below is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project’s own docs or release notes as of 2026-10-05.

Mode key: S is standalone YAML, K is Kubernetes AgentgatewayPolicy promptGuard, S+K is both. Field names differ. Grades are Tested only where Marketing ran the open-source standalone binary v1.6.0 on 2026-10-05; everything else is Documented.

What AI gateway guardrails are

AI gateway guardrails are content-safety checks an AI or agent gateway runs on LLM requests and responses, and, when you configure it, on tool-related fields. Each check can allow the content through, reject it, mask matched spans, or audit the match without changing traffic. The gateway is the enforcement point, so every client behind it inherits the same policy.

client
  |
  v
agentgateway
  1. regex (fast, deterministic)
  2. optional external guard (Bedrock, Model Armor, OpenAI Moderation, Azure Content Safety)
  3. optional custom webhook
  |
  v
LLM
  |
  v
agentgateway  (response guards, same layers)
  |
  v
client

Guards run only on LLM conversation routes: completions, messages, responses, and generateContent (S+K, Documented). realtime runs only when streaming is enabled, and mask still does not rewrite WebSocket frames. Embeddings, rerank, token-count, models, passthrough, and detect skip guards. Authorization still sits in front; two CEL authorization gotchas can fail open or 403 every call before a guard runs.

agentgateway’s guardrail layers

The standalone overview and the Kubernetes overview describe the same three layers (S+K, Documented):

Mask is a regex capability. External guards and webhooks reject or audit. A webhook can still return its own mask verdict (the sample server turns the word mask into ****).

Default rejection is HTTP 403 with the body The request was rejected due to inappropriate content (S+K, Documented). Standalone customizes that with rejection.status, rejection.body, and rejection.headers. Kubernetes uses response.message and response.statusCode. Webhooks ignore those fields and set their own status and body.

On standalone, llm.policies.guardrails is a shared baseline for every model, and llm.models[].guardrails adds per-model guards. The two lists merge. Kubernetes attaches promptGuard on an AgentgatewayPolicy that targets a route.

Regex filters: builtins, custom patterns, mask vs reject

Standalone regex and Kubernetes regex ship the same builtins with different names (S / K, Documented):

Custom patterns are first-class. Standalone uses rules with pattern. Kubernetes uses regex.matches. The Kubernetes docs include example sets for instruction-override phrases, credentials, and jailbreak framing. Those examples are Documented, not a production pack I tested.

Request reject, both modes

Marketing’s standalone harness (P2, Tested) rejects an email with a custom 400 body (code: pii_detected) and rejects the keywords SSN / Social Security with code: content_policy_violation. A clean "Hello" reaches the mock upstream at HTTP 200.

Standalone (S, Tested on v1.6.0). A second request rule used pattern: SSN / Social Security and code: content_policy_violation:

guardrails:
  request:
  - regex:
      action: reject
      rules:
      - builtin: email
    rejection:
      status: 400
      headers:
        set:
          content-type: "application/json"
      body: '{"error":{"message":"Request blocked: Contains email address","type":"invalid_request_error","code":"pii_detected"}}'

Curl outcomes from that run:

POST /v1/chat/completions  {"messages":[{"role":"user","content":"Hello"}]}
-> HTTP 200   mock assistant reply

POST ...  "Contact me at [email protected]"
-> HTTP 400   {"error":{"code":"pii_detected", ...}}

POST ...  "My SSN is on file"
-> HTTP 400   {"error":{"code":"content_policy_violation", ...}}

The Kubernetes twin is the same policy under AgentgatewayPolicy (K, Documented; P8 was not run):

apiVersion: agentgateway.dev/v1alpha1
kind: AgentgatewayPolicy
metadata:
  name: regex-request-reject
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: openai
  backend:
    ai:
      promptGuard:
        request:
        - regex:
            action: Reject
            builtins: [Email]
          response:
            message: "Request blocked: Contains email address"
            statusCode: 400
        - regex:
            action: Reject
            matches: ["SSN", "Social Security"]
          response:
            message: "Request rejected: Content contains sensitive information"
            statusCode: 400

Response mask, both modes

Standalone response mask on builtin phoneNumber (S, Tested, P3, stream: false). The mock returned 919 222 1111. The assistant content came back with <PHONE_NUMBER> and the raw digits were gone. The access log recorded phase":"response","guard":"regex","action":"mask".

guardrails:
  response:
  - regex:
      action: mask
      rules:
      - builtin: phoneNumber

Kubernetes (K, Documented):

      promptGuard:
        response:
        - regex:
            action: Mask
            builtins:
            - PhoneNumber

Mask on a buffered response is the path that works. Mask on a stream is the F8 hole below.

Custom webhooks

A webhook is how you bring your own detector. agentgateway calls it synchronously on the request path and, if configured, on the response path.

Standalone (webhooks , S, Documented; Tested in P5): webhook.target.host is a host:port or an http(s) URL. Defaults are POST /request and POST /response. Optional headers is a CEL map (including :path). Default timeout is 10 seconds; override with target.policies.http.requestTimeout or a named backend. failureMode is failClosed unless you set failOpen.

guardrails:
  request:
  - webhook:
      target:
        host: "127.0.0.1:18081"
      failureMode: failClosed
  response:
  - webhook:
      target:
        host: "127.0.0.1:18081"
      failureMode: failClosed

Kubernetes (K, Documented): webhook.backendRef points at a Service. The webhook guide deploys a sample gcr.io/solo-public/docs/ai-guardrail-webhook:latest that rejects block and masks mask to ****. Timeout is a second AgentgatewayPolicy with backend.http.requestTimeout on that Service. Pooling knobs are GUARDRAIL_WEBHOOK_MAX_CONNECTIONS and GUARDRAIL_WEBHOOK_KEEPALIVE_SECONDS on AgentgatewayParameters.

      promptGuard:
        request:
        - webhook:
            backendRef:
              kind: Service
              name: ai-guardrail-webhook
              port: 8000
        response:
        - webhook:
            backendRef:
              kind: Service
              name: ai-guardrail-webhook
              port: 8000

Guard-level rejection / response.message is ignored for webhooks. The webhook owns the status and body.

P5 against a local sample server (S, Tested): a clean prompt was allowed (HTTP 200). A prompt containing block returned HTTP 403 with the webhook body request blocked by harness webhook. A request containing mask logged action":"mask". A response containing mask came back with **** in the assistant text.

AWS Bedrock Guardrails and Google Model Armor

Both attachments are model-agnostic. They apply to request and/or response, regardless of where the upstream model is hosted (S+K, Documented). Marketing had no cloud credentials or pre-created guardrail or template, so P7 is SKIP, Documented OK. Nothing below is Tested.

Standalone Bedrock (docs ) uses guardrailIdentifier, guardrailVersion, region, and policies.backendAuth.aws. Kubernetes (docs ) uses identifier, version, region, and policies.auth.aws (secretRef or {}). Optional http.requestTimeout sits under policies.

# Standalone (S, Documented)
- bedrockGuardrails:
    guardrailIdentifier: <your-guardrail-id>
    guardrailVersion: DRAFT
    region: us-west-2
    policies:
      backendAuth:
        aws: {}

# Kubernetes (K, Documented)
- bedrockGuardrails:
    identifier: <guardrail-ID>
    version: "DRAFT"
    region: us-west-2
    policies:
      auth:
        aws:
          secretRef:
            name: aws-secret

Bedrock is one of two guard types that accept a non-default scope (the other is regex). A Social Security number that arrives only in a tool result is invisible unless scope lists toolOutput.

Standalone Model Armor (docs ) takes templateId, projectId, location, and policies.backendAuth.gcp. Kubernetes (docs ) uses the same identifiers under promptGuard, with policies.auth.gcp (for example type: AccessToken) and documented kind or GKE workload-identity setup.

# Standalone (S, Documented)
- googleModelArmor:
    templateId: <your-template-id>
    projectId: <your-project-id>
    location: us-central1
    policies:
      backendAuth:
        gcp: {}

# Kubernetes (K, Documented)
- googleModelArmor:
    templateId: <template-ID>
    projectId: <project-ID>
    location: us-central1
    policies:
      auth:
        gcp:
          type: AccessToken

OpenAI Moderation is in the same external family. Azure Content Safety is listed on the standalone overview. Both stay Documented; this page does not walk them.

Streaming, failure modes, and scope

Streaming (F8)

Default: response guards do not run when the client sets "stream": true (S+K, Documented). Enable them with streaming: Enabled on standalone (llm.policies.guardrails.streaming or per-model) or promptGuard.streaming: Enabled on Kubernetes.

Mask cannot rewrite content that is already on the way to the client. A matching response action: mask (or Mask) on a stream passes that content through unmodified. No client error. No guardrail event. The same limit applies to a webhook mask and to Bedrock anonymization on a stream. Request guards do not have this hole: the request is buffered, so mask and reject still apply.

P4a (S, Tested, F8 honesty): streaming unset (default Disabled), response action: mask on phoneNumber, client "stream": true. SSE HTTP 200. Raw 919 222 1111 in the stream. No <PHONE_NUMBER> placeholder. No response-phase agw.ai.guardrails action in the access log.

P4b (S, Tested): streaming: Enabled, response action: reject on phoneNumber, client "stream": true. The SSE response stayed HTTP 200. The stream ended with an in-band error:

data: {"error":{"type":"guardrail_blocked","code":"guardrail_intervention","message":"...pii_detected..."}}

Raw phone digits were absent. The log recorded action":"reject". To protect a streamed response, enable streaming and reject. Do not rely on mask.

failureMode (F6)

failureMode applies to webhook, OpenAI Moderation, Bedrock Guardrails, Google Model Armor, and Azure Content Safety (S+K, Documented). Default failClosed / FailClosed rejects when the provider is unreachable or returns an error. failOpen / FailOpen lets the content continue.

A provider error is not a verdict. action: audit (or Audit) still incurs provider latency and cost, and it does not change error handling. An audit guard with the default failureMode still rejects when the provider call fails.

Scope (F7)

A request guard does not read the whole request. Default scope is systemPrompt + messages (S), or SystemPrompt + Messages (K). Tool results pass unless scope includes toolOutput / ToolOutput.

Scope replaces the default. It does not add to it. scope: [toolOutput] reads tool results and stops reading messages. To cover both, list both values. An empty list is rejected.

Only regex and bedrockGuardrails accept a non-default scope. Any other guard type fails to load if scope is set. The field applies to request guards only. Masking toolInput can produce invalid JSON, because chat-completions tool arguments are one opaque string.

# Standalone (S, Documented)
request:
- scope: [systemPrompt, messages, toolOutput]
  regex:
    action: mask
    rules:
    - builtin: ssn

# Kubernetes (K, Documented)
request:
- scope:
  - SystemPrompt
  - Messages
  - ToolOutput
  regex:
    action: Mask
    builtins:
    - Ssn

Proof: Marketing-owned harness (v1.6.0)

Marketing ran the open-source standalone linux-amd64 binary on 2026-10-05 09:19:35 PDT. Release v1.6.0 . Reported version: 1.6.0, git_revision: ea5608642b9d5c94c5baf5fd9f2f9849b808e963. Checksum 496ed27e6a96b55a9ad7cabe5c2be7172511f44a3cb113e533d22bf93a7f079d (sha_match=true). Upstream: local mock POST /v1/chat/completions on 127.0.0.1:18080. Listener http://127.0.0.1:4000. No provider key, Docker, Kubernetes, or cloud guardrail account.

The harness is Marketing box artifacts (README labelled v1.6.0, test date above). It is not a public clone in this repo. This PR does not publish the binary.

P6 excerpt, request reject (email):

http.status=400 ... agw.ai.guardrails=[{"phase": "request", "guard": "regex", "action": "allow"}, {"phase": "request", "guard": "regex", "action": "reject"}] error="request rejected by regex guardrail" reason=Guardrail

P3 excerpt, response mask: agw.ai.guardrails=[{"phase": "response", "guard": "regex", "action": "mask"}]. P4a logged http.status=200 and did not record a response-phase guardrail action. That is the F8 honesty note, not a missed scrape.

Not the WAF page (and not egress)

A WAF that reads the prompt is Solo Enterprise for agentgateway: Coraza plus the OWASP Core Rule Set, HeadersAndBody, a JSON body processor, and custom SecLang on LLM prompts and MCP JSON-RPC. The CRDs are WAFPolicy and EnterpriseAgentgatewayPolicy. That lab ran on v2026.8.2. It is HTTP and signature inspection, not OSS prompt guards.

This page is the OSS companion: regex, webhook, Bedrock Guardrails, Model Armor, on LLM conversation routes. A CRS 403 is not proof that promptGuard fired.

Egress is a different control. A CONNECT hostname allowlist decides where an agent may go, not what a prompt contains.

Per-key dollar and token budgets are not this page; see per-key LLM budgets (OSS standalone) and the labelled Solo Enterprise cost-controls walkthrough .

The scorecard that lists content safety as one criterion is how to choose an AI agent gateway . This page does not re-score seven controls.

Methodology and disclosure

Last checked 2026-10-05, agentgateway docs labelled latest (standalone prompt-guards and Kubernetes guardrails). Releases through v1.6.0 (published 2026-10-02). I fetched the standalone overview, regex, webhooks, Bedrock Guardrails, and Model Armor pages, and the Kubernetes overview, regex, webhook, Bedrock Guardrails, and Model Armor pages. The live sibling WAF for LLM and MCP traffic was skimmed for the Solo Enterprise Coraza / CRS differentiation.

Proof is Marketing’s 2026-10-05 standalone harness, v1.6.0, git ea560864…, checksum verified, test date 2026-10-05 09:19:35 PDT. P1 through P6 and the P4a/P4b pair are Tested (S). P7 and P8 are Documented only. I am not claiming readers can clone that harness from this repository. Configs and curl outcomes above are copied from those Marketing box artifacts.

Every agentgateway claim is marked S, K, or S+K. Standalone field names are not the Kubernetes names. Solo Enterprise appears only as a label on the WAF sibling and on the Kubernetes dollar-budget page. I have not invented recall scores or adoption counts, and I have not recommended another gateway or a third-party guardrail vendor.

Disclosure: I work at Solo.io, which created agentgateway and sells Solo Enterprise for agentgateway. Everything above is open-source agentgateway unless it is labelled Solo Enterprise, and every claim links to the project’s own docs or release notes as of 2026-10-05.