LLM guardrails compared: what the platforms catch and what you still have to build

Every major platform now sells a guardrail layer, and in 2026 they are genuinely better than they were a year ago. Prompt-attack classifiers have become a standard feature rather than a premium add-on, grounding checks have moved past the demo stage, and Microsoft and OpenAI now let you put checks on agent tool calls, not just on chat messages.

What hasn't changed is the gap between what a guardrail product checks and what a production system needs. The dangerous failures in our audits are rarely “the classifier missed a rude word.” They are an agent that read a poisoned web page and emailed out a customer list, a PII filter that never saw tool arguments, or a grounding check quietly switched off because it blocked too many good answers.

This guide covers the layered model we use, what each managed and open-source option actually does as of October 2026, where each one stops, and how to test and budget for the whole stack.

The layered model: eight jobs, not one product

“Guardrails” is used loosely. It helps to split it into the separate jobs a production system has to do, because each job has a different best tool and a different failure mode. Most of them map onto the OWASP Top 10 for LLM Applications (2025 edition).¹

Managed guardrail services mostly cover layers 1, 2, 5 and 6. Layers 3, 4, 7 and 8 are application architecture, and they are where the serious incidents come from. Keep that split in mind when you read the vendor sections below.

What the managed services actually do

This section is based on each vendor's current documentation. Preview labels change often, so check the linked page before designing around a feature.

Amazon Bedrock Guardrails

Bedrock offers the broadest set of policies in a single configurable object: content filters (Hate, Insults, Sexual, Violence, Misconduct and Prompt Attack), denied topics, exact-match word filters, sensitive information filters, contextual grounding checks and Automated Reasoning checks.² You can attach a guardrail to Bedrock model calls, or call the standalone ApplyGuardrail API to check text from any source.

The details that matter in production:

• The prompt-attack filter only sees what you tag. With InvokeModel you must wrap user input in guard tags, or prompt attacks are not filtered at all. The docs also say the filter does not evaluate tool results or tool definitions, which is exactly where indirect injection lives in an agent.³
• Pick the tier on purpose. The Standard tier adds prompt-leakage detection, broader language support and coverage of harmful content inside code. It also requires cross-Region inference. Classic supports English, French and Spanish only.⁴
• PII filtering skips tool traffic and logs. The sensitive information filter can block or mask built-in PII types plus your own regexes. But it does not evaluate tool-call arguments the model generates, tool results, or tool definitions. And if model invocation logging is on, the logged input is the original, unmasked request.⁵
• Grounding has hard size limits. Contextual grounding returns separate grounding and relevance scores against thresholds you set between 0 and 0.99. The source can be up to 100,000 characters, the query 1,000 and the response 5,000. With streaming, an irrelevant response may only be flagged after it has already been streamed to the user.⁶
• Automated Reasoning checks are a verifier, not a filter. They translate a policy document into formal logic and check whether responses are consistent with it. They are generally available in a handful of Regions, run in detect mode only, support English (US) only, do not support streaming, and AWS says plainly that they provide no prompt-injection protection.⁷ They are well suited to rule-heavy domains like benefits eligibility. They are not a general hallucination fix.

Azure AI Content Safety and Foundry guardrails

Azure splits the same ground into separate APIs. There are harm classifiers for text and images (hate, sexual, violence, self-harm, each with severity levels), Prompt Shields, groundedness detection (preview), protected material detection, a task adherence API for agent tool use, and custom categories (preview).⁸

Prompt Shields is the part most teams come for. It separates user prompt attacks from document attacks, meaning instructions hidden in emails, web pages or retrieved files. The standalone API takes a user prompt plus up to five documents and returns a detection result. It does not block anything; your code decides what to do. Microsoft is explicit that Prompt Shields do not judge whether content is harmful, grounded or on-task. It also offers Spotlighting (preview), which base64-encodes untrusted documents so the model treats them as less trusted. That costs extra tokens, and the model sometimes mentions the encoding in its answer.⁹

In Microsoft Foundry these are packaged as guardrails made of controls. For agents (in preview), each control can apply at four intervention points: user input, tool call, tool response and output. That is the closest any managed service comes to inspecting the agent loop itself. The caveats: agent guardrails only cover agents built in Foundry Agent Service, groundedness and Spotlighting are not yet supported for agents, and document-attack detection is off by default.¹⁰⁹ Groundedness detection is English-only and offers Medical and Generic domains, with an optional correction feature that rewrites ungrounded text.¹¹

Google Cloud Model Armor

Model Armor is Google's screening service for prompts and responses. It has responsible-AI filters (hate speech, harassment, sexually explicit, dangerous content), an always-on CSAM filter, prompt-injection and jailbreak detection, Sensitive Data Protection for PII and secrets, and malicious URL detection. Each filter takes a confidence threshold, and you can run in inspect-only mode before switching to inspect-and-block. Organization-level floor settings let a security team set a minimum that individual projects can't weaken. Google says it applies whether your models run on Google Cloud or elsewhere, and lists integrations with Vertex AI, Apigee, Google's own MCP servers and LangChain.¹²

The documented limits are worth knowing. Each prompt and response is evaluated as a single turn with no conversation history, encoded content such as Base64 is not decoded, and prompt-injection detection returns no match for inputs under three words.¹² The first point matters most. A multi-turn attack that builds up slowly will not show up in any single message.

OpenAI: moderation, Agents SDK guardrails and the Guardrails library

OpenAI ships three different things. The moderation endpoint (omni-moderation-latest) is free, accepts text and images, and scores 13 harm categories. OpenAI notes that the scores are signals for your own policy, and that thresholds may need recalibrating when the underlying model is updated.¹³

The Agents SDK has input guardrails, output guardrails and tool guardrails that wrap individual function tools. One default deserves attention: input guardrails run in parallel with the agent. That saves latency, but the agent may already be spending tokens or calling tools when the tripwire fires. Blocking mode avoids this. Tool guardrails also do not apply to hosted or built-in tools.¹⁴

The open-source OpenAI Guardrails library adds ready-made checks for jailbreaks, prompt injection, PII, moderation, URL filtering, off-topic prompts and hallucination, run in pre-flight, input and output stages.¹⁵ Its Prompt Injection Detection check compares tool calls and tool outputs against the user's goal. Its documentation is unusually candid. With the default model, recall at a 1% false-positive rate is reported as zero on their benchmark, against 0.584 with gpt-5. Median latency ranges from about 1.5 seconds with the default model to about 4 seconds with gpt-5.¹⁶ That is the general trade-off for LLM-based checks: they understand intent better than small classifiers, but you pay for it in seconds per step.

Side by side

This is not a ranking. Each covers a different slice, and none covers layers 3, 7 or 8 for you.

Open-source building blocks

If you need to self-host, run across clouds, or tune detection to your own traffic, the open-source options are now mature enough to use in production:

• NVIDIA NeMo Guardrails is an orchestration framework with five rail types (input, dialog, retrieval, execution for tool calls, and output), configured in Colang. NVIDIA itself warns that the built-in rails may not suit every production use case.¹⁷
• Guardrails AI combines validators from its Hub into input and output guards, and is strong at enforcing structured output against a Pydantic schema. In 2026 its validators moved to standard PyPI packages and hosted remote inference was discontinued, so older setups need migrating.¹⁸
• Llama Guard 4 is a 12B multimodal classifier for prompts and responses, built on the MLCommons hazard taxonomy. Its model card notes it may itself be vulnerable to adversarial attacks.¹⁹
• Llama Prompt Guard 2 comes in 86M and 22M sizes, has a 512-token window (split longer inputs), and only flags explicit attempts to override instructions. Meta recommends fine-tuning it on your own data.²⁰ It is one component of LlamaFirewall, alongside AlignmentCheck (an LLM auditor that looks for goal hijacking in an agent's reasoning) and CodeShield (static analysis of generated code).²¹
• gpt-oss-safeguard (20B and 120B, Apache 2.0) is an open-weight reasoning model that classifies content against a written policy you supply at inference time. That makes it useful for policies too specific for any off-the-shelf category list.²²

The usual pattern is a small, fast classifier on every request and a slower LLM-based or policy-reasoning check only where the risk justifies the latency.

Indirect prompt injection: why classifiers are not the control

Indirect prompt injection means an attacker plants instructions in content the system will read later: a web page, an email, a PDF, a support ticket. It was demonstrated against real applications in 2023.²³ For agents it is the risk that matters most, because the injected text arrives with the agent's permissions attached.

Every injection classifier above helps, and every one can be beaten. In October 2025, researchers from OpenAI, Anthropic, Google DeepMind and several universities took 12 published defenses and attacked them adaptively. They bypassed most with attack success rates above 90%, even though most had originally reported near-zero.²⁴ As Simon Willison puts it, a filter that catches 95% of attacks is a failing grade in security.²⁵

So the useful question is not “will the classifier catch it?” but “what can the agent do if it doesn't?” Two simple frameworks help. Willison's lethal trifecta: an agent that has private data, reads untrusted content, and can communicate externally can be made to leak that data.²⁵ Meta's Agents Rule of Two: in a single session, an agent should have no more than two of untrusted input, access to sensitive data, and the ability to change state or communicate externally. If it needs all three, a human should be in the loop.²⁶

Architectural controls that hold up

• Privilege separation. Content from untrusted sources is read by a model with no tools. It returns typed data (a category, an amount, an extracted field), never free text that ends up back in the privileged agent's instructions. This is the dual-LLM idea, one of several patterns in a 2025 paper on securing agents by design.²⁷
• Plan before reading. Fix the sequence of tool calls from the trusted user request before any untrusted content is read, so injected text can change data but not control flow. CaMeL, from researchers at Google, Google DeepMind and ETH Zurich, takes this furthest. It tracks where every value came from and enforces policies at tool-call time, completing 77% of AgentDojo tasks with provable security, compared with 84% undefended.²⁸
• Allow-listed tools with narrow arguments. send_email(to, body) with any recipient is an exfiltration channel. reply_to_ticket(ticket_id, body) is not.
• Authorization in the tool, not the prompt. The tool checks the end user's permissions on every call. The model never holds credentials broader than the user it is acting for.
• Confirmation for irreversible actions. Show the human the actual action and arguments, not the model's summary of them. Meta notes that even this fails if users click through warnings without reading them, so keep confirmations rare enough that people still read them.²⁶
• Egress control. Restrict which domains the agent can fetch from and post to. Watch for data leaving through rendered markdown images and links as well.

In code, this is a policy check that runs on every tool call, outside the model:

TOOL_POLICY = {
    "search_kb":       {"risk": "read",  "confirm": False},
    "get_order":       {"risk": "read",  "confirm": False, "scope": "own_orders"},
    "reply_to_ticket": {"risk": "write", "confirm": False, "scope": "own_ticket"},
    "issue_refund":    {"risk": "money", "confirm": True,  "max_amount": 200},
}

def authorize(call, session):
    policy = TOOL_POLICY.get(call.name)
    if policy is None:
        return deny("tool not allow-listed")

    # Rule of Two: once untrusted content is in context,
    # anything that changes state needs a human.
    if session.tainted and policy["risk"] != "read":
        return require_confirmation(call)

    if policy.get("scope") == "own_orders":
        if not session.user.owns(call.args["order_id"]):
            return deny("order not owned by caller")

    if call.name == "issue_refund":
        if call.args["amount"] > policy["max_amount"]:
            return escalate_to_human(call)

    return require_confirmation(call) if policy["confirm"] else allow()

The tainted flag gets set whenever a tool returns content from outside your trust boundary: web results, inbound email, user-uploaded files. Classifiers still run, but they become an early warning rather than the only thing in the way.

Mapping to the OWASP Top 10 for LLM Applications

The 2025 OWASP list is a useful coverage check.¹ Here is where guardrail products help, and what still has to be built by your team:

For agents, OWASP has also published a separate Top 10 for Agentic Applications (December 2025). It is worth reading alongside this list if your system takes actions.²⁹

A worked example: a support agent with refunds

Here is a composite of a design we see often: a customer-support agent that answers from a knowledge base, reads the customer's ticket history, and can issue small refunds. Layer by layer:

1. Input: the platform harm filter and injection classifier run on the user message, in blocking mode, at a per-user rate limit.
2. Retrieval: knowledge-base articles are trusted. Ticket history is not, because customers wrote it. Ticket text goes through document-attack detection and sets the tainted flag.
3. Tools: the policy above. Once the session is tainted, refunds need the customer to confirm the exact amount in the UI. Anything over the limit goes to a human queue.
4. Output: the response must validate against a schema with answer, citations and action fields. A grounding check runs against the cited articles, and PII is masked before display and before logging.
5. Telemetry: every guardrail decision goes into the trace with its layer, score and action.

On Bedrock, layers 1, 2, 5 and 6 can be one guardrail. A trimmed configuration, using field names from the AWS docs, looks like this:³⁵⁶

{
  "name": "support-agent-prod",
  "contentPolicyConfig": {
    "filtersConfig": [
      { "type": "PROMPT_ATTACK", "inputStrength": "HIGH", "outputStrength": "NONE" },
      { "type": "MISCONDUCT", "inputStrength": "MEDIUM", "outputStrength": "MEDIUM" }
    ],
    "tierConfig": { "tierName": "STANDARD" }
  },
  "sensitiveInformationPolicyConfig": {
    "piiEntitiesConfig": [
      { "type": "EMAIL", "action": "ANONYMIZE" },
      { "type": "CREDIT_DEBIT_CARD_NUMBER", "action": "BLOCK" }
    ],
    "regexesConfig": [
      { "name": "account_id", "pattern": "ACC-[0-9]{8}", "action": "ANONYMIZE" }
    ]
  },
  "contextualGroundingPolicyConfig": {
    "filtersConfig": [
      { "type": "GROUNDING", "threshold": 0.7, "action": "NONE" },
      { "type": "RELEVANCE", "threshold": 0.7, "action": "NONE" }
    ]
  },
  "crossRegionConfig": { "guardrailProfileIdentifier": "<profile-id>" }
}

Note the grounding action of NONE. That is detect mode: we log scores for a few weeks of real traffic before letting the check block anything. The regex handles an internal identifier no built-in PII type knows about. And because the PII filter ignores tool arguments, the refund tool redacts its own logs.

Testing guardrails like any other classifier

A guardrail is a classifier with a false-negative rate and a false-positive rate, and most teams measure neither. False positives are what kill guardrails in practice: when legitimate users get blocked, someone lowers the threshold or turns the check off, often without telling security.

• Start in detect mode. The managed services support it: Bedrock's “Detect (no action)”, annotate-only for Azure model deployments, inspect-only in Model Armor.³⁹¹² Run for two to four weeks and look at what would have been blocked.
• Measure false positives on your own traffic. Sample a few hundred real requests the check flagged and a few hundred it didn't, label them, and compute precision and recall per category. A medical, legal or security product will see far more benign “dangerous” content than vendor benchmarks assume.
• Keep a red-team set and run it on every change. Include direct jailbreaks, injections hidden in each document type you ingest, encoded payloads, multi-turn build-ups, and your own past incidents. Version it alongside the guardrail config.
• Test the architecture, not just the filter. Plant an injection in a test ticket that tells the agent to refund a different order or email a summary to an outside address, and check that the tool policy stops it even when the classifier is switched off.
• Re-test when anything upstream changes. Vendors update classifier models, and a tier change can shift grounding scores. AWS explicitly tells you to re-test thresholds when you change tiers.⁶

Latency and cost: measure, don't guess

We won't quote a single latency number for any of these services, because it depends on region, input length, tier and which policies are on. What we can say from vendor documentation is that the spread is large. Small classifiers such as Prompt Guard 2 (86M or 22M parameters) are cheap to run on every request, and the 22M variant cuts latency and compute by about 75% versus the 86M one.²⁰ LLM-based checks are measured in seconds, as the OpenAI figures above show.¹⁶

Three design choices matter more than which vendor you pick:

• Parallel versus blocking. Running input checks in parallel with the model hides their latency. It is only safe if nothing irreversible can happen before the check returns.¹⁴
• Streaming. Output checks either buffer the response, which adds time to first token, or check chunks as they stream and accept that users may see text that later gets withdrawn. Some checks don't support streaming at all.⁷⁶
• Agent loops multiply everything. A ten-step agent with checks on every tool call and response runs those checks twenty times per task. Put expensive checks only where the trust boundary is crossed.

Record each layer as its own span in your traces and read p50 and p95 per layer from real traffic. Do the same for cost: managed checks are billed per call or per unit of text, and agents multiply the call count.

Failure modes we see most

• Checking the chat, not the agent. Filters on the user message and final answer, nothing on tool results, which is where injected content arrives.
• Untagged input. Prompt-attack filters configured but never applied, because user content wasn't marked up the way the API expects.
• PII in the logs. The response is masked while the raw prompt sits in invocation logs and the trace store.
• Silent fail-open. The guardrail call times out, the request goes through, and nothing records it.

Checklist

1. Map your system against all eight layers and write down which product or code owns each one.
2. Label every input source as trusted or untrusted, and track taint through the session.
3. Apply the Rule of Two. Any agent with untrusted input, sensitive data and outbound actions gets a human confirmation step.
4. Enforce tool authorization and argument limits in code, outside the model.
5. Validate every model output against a schema before anything downstream uses it.
6. Run PII redaction on prompts, responses, tool arguments and logs, with regexes for your own identifier formats.
7. Start every new check in detect mode and measure its false-positive rate on real traffic before enabling blocking.
8. Keep a versioned red-team set, including indirect injections in every document type you ingest, and run it in CI.
9. Decide whether each guardrail fails open or closed, and alert when it fails.
10. Trace every guardrail decision with its latency, and review both blocked and allowed samples weekly.

If you want an outside view of how your agents would hold up against these attacks, this is the kind of testing our AI security work covers. We start with the architecture, not the filter settings.

Sources

1. OWASP GenAI Security Project, “OWASP Top 10 for LLM Applications 2025” — genai.owasp.org/llm-top-10/
2. AWS, Amazon Bedrock User Guide — “Create your guardrail” (guardrail components) — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-components.html
3. AWS, Amazon Bedrock User Guide — “Detect prompt attacks with Amazon Bedrock Guardrails” — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-prompt-attack.html
4. AWS, Amazon Bedrock User Guide — “Safeguard tiers for guardrails policies” — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-tiers.html
5. AWS, Amazon Bedrock User Guide — “Remove PII from conversations by using sensitive information filters” — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-sensitive-filters.html
6. AWS, Amazon Bedrock User Guide — “Use contextual grounding check to filter hallucinations in responses” — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-contextual-grounding-check.html
7. AWS, Amazon Bedrock User Guide — “What are Automated Reasoning checks in Amazon Bedrock Guardrails?” — docs.aws.amazon.com/bedrock/latest/userguide/guardrails-automated-reasoning-checks.html
8. Microsoft Learn, “What is Azure AI Content Safety?” — learn.microsoft.com/en-us/azure/ai-services/content-safety/overview
9. Microsoft Learn, “Prompt Shields in Azure AI Content Safety” — learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection
10. Microsoft Learn, “Guardrails and controls overview in Microsoft Foundry” — learn.microsoft.com/en-us/azure/foundry/guardrails/guardrails-overview
11. Microsoft Learn, “Groundedness detection in Azure AI Content Safety” — learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/groundedness
12. Google Cloud, “Model Armor overview” — docs.cloud.google.com/security-command-center/docs/model-armor-overview
13. OpenAI API docs, “Moderation” guide — developers.openai.com/api/docs/guides/moderation
14. OpenAI Agents SDK (Python) docs, “Guardrails” — openai.github.io/openai-agents-python/guardrails/
15. OpenAI Guardrails (Python) documentation — openai.github.io/openai-guardrails-python/
16. OpenAI Guardrails (Python) docs, “Prompt Injection Detection” check reference — openai.github.io/openai-guardrails-python/ref/checks/prompt_injection_detection/
17. NVIDIA, NeMo Guardrails repository (NVIDIA-NeMo/Guardrails) — github.com/NVIDIA-NeMo/Guardrails
18. Guardrails AI, guardrails repository — github.com/guardrails-ai/guardrails
19. Meta, Llama Guard 4 (12B) model card — huggingface.co/meta-llama/Llama-Guard-4-12B
20. Meta, Llama Prompt Guard 2 (86M) model card — huggingface.co/meta-llama/Llama-Prompt-Guard-2-86M
21. Chennabasappa et al. (Meta), “LlamaFirewall: An open source guardrail system for building secure AI agents,” arXiv:2505.03574, May 2025 — arxiv.org/abs/2505.03574
22. OpenAI, gpt-oss-safeguard-20b model card — huggingface.co/openai/gpt-oss-safeguard-20b
23. Greshake et al., “Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection,” arXiv:2302.12173, 2023 — arxiv.org/abs/2302.12173
24. Nasr et al., “The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections,” arXiv:2510.09023, Oct 2025 — arxiv.org/abs/2510.09023
25. Simon Willison, “The lethal trifecta for AI agents: private data, untrusted content, and external communication,” 16 Jun 2025 — simonwillison.net/2025/Jun/16/the-lethal-trifecta/
26. Meta AI, “Agents Rule of Two: A Practical Approach to AI Agent Security,” 31 Oct 2025 — ai.meta.com/blog/practical-ai-agent-security/
27. Beurer-Kellner et al., “Design Patterns for Securing LLM Agents against Prompt Injections,” arXiv:2506.08837, Jun 2025 — arxiv.org/abs/2506.08837
28. Debenedetti et al., “Defeating Prompt Injections by Design” (CaMeL), arXiv:2503.18813, 2025 — arxiv.org/abs/2503.18813
29. OWASP GenAI Security Project, “OWASP Top 10 for Agentic Applications for 2026,” 9 Dec 2025 — genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/

Product capabilities were checked against each vendor's documentation in October 2026. Preview features and limits change often, so confirm against the linked page before relying on one.