Would we notice if the system started producing nonsense, or being abused?
Enhanced: 22 controls
Who it applies to
Enter for every AI system whose letter is E or A (this area has no Baseline controls). Go as deep as your system letter, and implement in your mode.
Why it exists
Detect the characteristic ways AI misbehaves in the moment — confidently stating falsehoods, refusing what it should do, being steered into abuse — because these are not the failures your existing monitoring was built to catch. A system can pass every uptime and performance check while telling users things that are simply untrue, or while being systematically manipulated. Watch for the behaviours that are specific to how AI fails, not only for whether it is running.
What to actually do
Watch for confident falsehood and anomalous behaviour. Put in place checks for the AI stating things that are untrue with apparent confidence, for outputs that fall outside expected patterns, and for signs the system is being pushed toward misuse. Where the stakes justify it, sample outputs for review by people who can tell right from plausible.
Route what you detect somewhere that acts on it. Detection that reaches no one changes nothing. Connect these signals to the incident detection below, to the guardrail tuning of Technical Security, and to the human overseers, so a detected misbehaviour becomes a correction rather than a log entry.
Controls
Enhanced2 controls
Control 203
Monitor for confident falsehood, anomalous output, and abuse patterns.
EnhancedDetectiveall rolesper system
What it is
With human review of samples where the stakes warrant.
How to implement it · Minimum rules
Confident falsehood: sampled outputs reviewed by someone who knows the subject; cited sources and figures checked to exist
Anomalous output: unusual length, language, format, refusal rate or topic compared with normal
Abuse: repeated attempts to override instructions, to extract hidden instructions or data, unusual volume per user
User reports: a "this is wrong" option or channel feeding the same review
Sample size and frequency scaled to the stakes, e.g. 20 outputs a week for customer-facing use
How to start
Start a weekly review of a small sample of real outputs by someone who knows the subject, and record what they find.
Evidence
The monitoring rules in place and the sampled review records.
Mode: light / standard / full
light
A weekly manual sample by a knowledgeable person, plus the provider's abuse-monitoring settings switched on.
standard
Automated checks for anomalies and abuse patterns, plus sampled review owned by the system owner.
full
Dedicated behavioural monitoring with automated scoring of outputs and an expert review team.
Control 204
Route detections into correction.
EnhancedCorrectiveall rolesper system
What it is
To incident handling, guardrail tuning, and human oversight.
How to implement it · Minimum rules
Each detection type has a destination: incident handling, guardrail tuning, or the human overseers
Severity rule: which detections open an incident rather than a tuning task
Every routed detection has an owner and a deadline
Closure records what was corrected, not only that it was seen
Recurring detections of the same kind trigger a review of the guardrails
How to start
Write a short routing table for this system: each type of detection, where it goes, and who owns the fix.
Evidence
The routing table and records showing detections that led to a correction.
Mode: light / standard / full
light
Detections emailed to the named owner and tracked in a spreadsheet until closed.
standard
Detections create tickets assigned by type in the ticketing system.
full
Automated routing to the incident, security and oversight functions, with closure rates reported to governance.
Evidence
The behavioural monitoring in place; sampled review records; evidence detections led to correction.