What stands between the model and harm while it is running?
Baseline: 2Enhanced: 3Advanced: 16 controls
Who it applies to
Enter for every AI system. Go as deep as your system letter, and implement in your mode.
Why it exists
Place automated checks around the model in live operation — and maintain them with the discipline you give your firewalls. Everything mandated so far acts before deployment; guardrails act in the moment, on every request, while the system runs. A guardrail is an automated check that inspects what goes into the model, what comes out of it, and what it is about to do — and blocks what should not pass. Two rules govern them. First, they are external to the model: a rule the model enforces on itself is a rule an attacker can talk it out of. Second, they decay: usage shifts, attacks evolve, and a guardrail nobody has revisited in a year is a control in name only. Set them deliberately, in layers, and put them on a maintenance cycle — exactly as you already do for the rules in your network firewalls.
What to actually do
Layer three kinds of check: on the way in, on the way out, and at the moment of action. On the way in, inspect what reaches the model — user input and retrieved documents alike — for hidden instructions and for sensitive data that should not be sent to it at all. On the way out, filter what the model produces before anyone sees it or anything consumes it: personal data, credentials, toxic content, and confident claims with no grounding. And at the moment of action, for AI that acts, check every intended action against what that system is permitted to do — the technical enforcement of the autonomy limits set in Governance — before it executes, not after. One layer is a speed bump; the three together are a control system, and attacks that slip one are caught by the next.
Never let the model be its own guardrail. An instruction to the model — “never reveal X”, “refuse requests to do Y” — is a request to a system whose defining property is that it can be persuaded. Every safety property you actually depend on must be enforced by a separate mechanism outside the model, one that holds even when the model itself has been successfully manipulated. Instructions set the model’s intended behaviour; external checks guarantee the boundaries of it. Confusing the two is the most common guardrail failure in deployed AI.
Review and tune the guardrails on a cycle, with numbers. Track how often each check triggers, what it caught, and what it wrongly blocked. A check that never triggers is either unneeded or broken — find out which. One that constantly blocks legitimate work will be disabled by the people it obstructs, and then it protects nothing. Review the set on a fixed cycle and after every incident, retiring, tightening, and adding checks as reality moves. You already run this exact discipline on firewall rules; extend it, unchanged, to these.
Decide now what happens when a guardrail is down. Checks fail, and dependencies go unavailable. Decide per system, in advance and in writing, whether it then stops or continues unprotected: a system whose errors can hurt people, money, or rights stops — it fails closed; a low-stakes convenience may continue degraded — it fails open. Both are legitimate; what is not legitimate is discovering, during the outage, that nobody ever decided.
Know the class of mechanism each layer needs. On the way in: a prompt-injection classifier and a personal-data detector. On the way out: an output classifier for personal data, credentials and toxic content, plus a groundedness check where the system answers from documents. At the moment of action: a policy check against the permitted-action list. Open-source and commercial implementations of each class exist; many are features of the AI gateway or the provider console.
Controls
Baseline2 controls
Control 118
Screen everything that reaches the model — user input and retrieved content alike.
BaselinePreventiveall rolesper system
What it is
Hidden instructions and out-of-policy sensitive data are caught on the way in.
How to implement it · Minimum rules
Cover every path into the model: user prompts, uploaded files, retrieved documents, tool and web results
Hidden instructions: "ignore previous instructions", role-play overrides, instructions in invisible text or file metadata
Sensitive data that must not be sent: identity numbers, bank and card numbers, credentials, health data, confidential documents
Action: block or mask, tell the user why, log the event
Start with the provider's built-in filters; add a separate check where they miss your sensitive data
How to start
Switch on the input filtering your provider or platform already offers, then test it with five known injection phrases and five samples of your sensitive data.
Evidence
The screening configuration and the log of blocked or masked inputs.
Mode: light / standard / full
light
Rely on the provider's built-in filters, with a named person checking the blocked-input log monthly.
standard
An AI gateway applies injection and personal-data detectors to every input, with events in the system log.
full
A dedicated guardrail service, tuned per system, screens all inputs and reports to security monitoring.
Control 119
Filter everything the model produces before it is seen or consumed.
BaselinePreventiveall rolesper system
What it is
Personal data, credentials, toxic content, and ungrounded claims are caught on the way out.
How to implement it · Minimum rules
Personal data not meant for the recipient: identity numbers, contact and bank details
Credentials and secrets: passwords, API keys and tokens, detected by pattern
Toxic content: harassment, hate, violence, self-harm, using the provider's content categories
Ungrounded claims: where the system answers from your documents, require a cited source and flag answers without one
Machine consumers: check format before another system uses the output (no code or commands where text is expected)
Action: block, redact or flag, and log the event
How to start
Turn on the provider's output filters for personal data, secrets and toxic content, and test each with a few prompts designed to trigger it.
Evidence
The output-filter configuration and the log of blocked or redacted outputs.
Mode: light / standard / full
light
Use the provider's built-in output filters, with a named person reviewing their log monthly.
standard
An AI gateway applies personal-data, secret and toxicity filters plus a groundedness check where documents are used.
full
A dedicated guardrail service filters all outputs and reports to security monitoring.
Enhanced3 controls
Control 120
Check every intended action against the system’s permissions before it executes.
EnhancedPreventiveall rolesper system
What it is
The runtime enforcement of the autonomy limits set in Governance.
How to implement it · Minimum rules
Permitted-action list per system, from its autonomy limits: tools, operations (read, create, send, pay, delete), targets and value limits
The check runs outside the model, before execution, on every tool call
Deny by default: any action not on the list is blocked
Actions above thresholds (amount, number of recipients, irreversibility) wait for human approval
Log every intended action with its decision: allowed, blocked or sent for approval
How to start
Write the permitted-action list for one acting system from its autonomy limits and configure its tool layer to refuse anything outside it.
Evidence
The permitted-action list, the action-check configuration and the decision log.
Mode: light / standard / full
light
Use the platform's built-in tool permissions and confirmation prompts, set from a one-page action list kept by the system owner.
standard
A policy check in the integration layer or AI gateway enforces each system's list.
full
A central policy service enforces action lists for every agent, with decisions streamed to security monitoring.
Control 121
Enforce every safety property you depend on outside the model.
EnhancedPreventiveall rolesper system
What it is
Instructions guide the model; external mechanisms guarantee the boundaries.
How to implement it · Steps
List every safety property the system depends on (e.g. never shows one customer's data to another, never sends payments)
For each, name the external mechanism that enforces it: access control, output filter, action check or data scoping
Remove secrets and credentials from system instructions; rely on access controls instead
Test each property with the model deliberately manipulated; it must still hold
Record any property enforced only by instruction as an open risk with owner and deadline
How to start
Find every "never" and "do not" in the system instructions and write next to each the external mechanism that enforces it, if any.
Evidence
The property-to-mechanism table and the results of the manipulation tests.
Mode: light / standard / full
light
The system owner keeps the table in a spreadsheet, using the provider's access and filter settings as mechanisms.
standard
The table is part of the system's design record and is checked at every release.
full
The security team maintains the table for all systems and verifies each property through adversarial testing.
Control 122
Review and tune the guardrail set on a fixed cycle, using trigger data.
EnhancedDetectiveall rolesper system
What it is
With records of what was caught, what was wrongly blocked, and what changed as a result.
How to implement it · Steps
Collect per check: trigger count, true catches, wrongful blocks, user complaints or overrides
Investigate checks that never trigger (unneeded or broken) and those with high wrongful-block rates
Decide per check: keep, tighten, loosen, retire, or add a new one, and record why
Review on a fixed cycle (e.g. quarterly) and after every incident
Apply the changes, re-test the affected checks and log what changed
How to start
Pull last quarter's trigger counts for every guardrail and hold a one-hour review with the system owner.
Evidence
The trigger and wrongful-block statistics, the review records and the change log.
Mode: light / standard / full
light
The system owner reviews the provider's filter statistics each quarter and notes decisions in a spreadsheet.
standard
The security team runs the review with gateway statistics and records changes through the change process.
full
A guardrail operations team tracks statistics on a dashboard, reviews continuously and runs a formal cycle with versioned rule sets.
Advanced1 controls
Control 123
Record a fail-closed or fail-open decision per system.
AdvancedCorrectiveall rolesper system
What it is
Made in advance, matched to the system’s stakes, and known to its operators.
How to implement it · Minimum content
System and each guardrail or dependency whose failure matters
Decision per failure: fail closed (stop) or fail open (continue degraded)
Rationale by stakes: possible harm to people, money or rights means fail closed
What users see and which fallback applies
Who is alerted, who may override and how the override is logged
Approver, date, next review, and confirmation that operators have been briefed
How to start
For each system, answer "if the guardrail service is down, does this stop or carry on?" and get the answer signed off.
Evidence
The signed fail-mode record per system and a test showing the system behaves as recorded.
Mode: light / standard / full
light
The owner-manager signs a one-line decision per system in the AI inventory spreadsheet.
standard
Decisions are recorded in each system's design record and tested before release.
full
Fail modes are enforced in the gateway configuration, tested in resilience exercises and covered in operator training.
Evidence
The guardrail configuration by layer; trigger and false-positive statistics; the review-cycle records and resulting changes; the per-system fail-mode decisions.