Area 2.9

Resource Abuse and Model Extraction

Can someone run up your bill, exhaust your service, or clone your model through the front door?

Baseline: 2Enhanced: 2Advanced: 2 6 controls

Who it applies to

Enter for every AI system. Go as deep as your system letter, and implement in your mode.

Why it exists

Every earlier dimension defends the AI against being manipulated; this one defends it against being over-used. An AI service billed by the request and reachable by many is exposed to a family of attacks that never touch the model's behaviour: flooding it with requests until the service falls over, or until the invoice does; crafting deliberately expensive queries that consume far more computation than they return; and — where the model itself has value — querying it systematically to reconstruct a copy, or harvesting its answers to train a functional clone. These are attacks on availability, on cost, and on the intellectual property embodied in the model, and conventional rate limits, written for ordinary web traffic, were never sized for how expensive a single AI request can be or how cloneable a model is.

The pay-per-use pricing of hosted AI is what turns a flood of requests into a financial attack: a botnet pointed at your assistant no longer merely slows it down, it spends your money — an outcome distinctive enough to have earned its own name, denial of wallet. Treat availability, cost, and model confidentiality as security properties here, not as billing concerns for someone else to notice at the end of the month.

What to actually do

Controls

Baseline 2 controls

Control 141

Enforce per-identity rate limits and usage quotas on every AI service.

BaselinePreventiveall rolesper system

What it is

Bound to the authenticated caller, not only to a network address

How to implement it · Minimum rules
  • Every caller is authenticated (user, application or key); no anonymous access to paid AI services
  • Rate limit per identity: requests per minute, set at a few times normal peak use
  • Quota per identity: requests or tokens per day or month
  • Separate keys per application or team, so usage can be attributed and limited
  • On limit: refuse with a clear message, log it, alert on repeated hits
How to start

Look at a month of usage per user or key and set limits at a few times the highest normal use.

Evidence

The rate-limit and quota configuration per identity and the log of limit hits.

Mode: light / standard / full
light
Use the provider's per-user or per-key limit settings and issue one key per application.
standard
The AI gateway enforces limits per authenticated identity, set by the system owner.
full
Limits are managed centrally through the identity and gateway platforms, with tiers per caller type and automated alerting.

Control 142

Bound input size, output length, and per-request cost, with timeouts on expensive calls.

BaselinePreventiveall rolesper system

What it is

A single request cannot consume resources out of proportion to its task

How to implement it · Minimum rules
  • Input: maximum length per request and maximum file size
  • Output: maximum tokens per response
  • Work per request: cap on tool calls, retrieved documents and agent steps or loops
  • Timeout: end calls that exceed a set duration and cap automatic retries
  • Cost: reject or throttle requests whose estimated cost exceeds a per-request ceiling
How to start

Set the provider's maximum-output setting and an input length limit in your application, based on the longest legitimate request you see.

Evidence

The configured input, output, work and cost bounds and the timeout settings.

Mode: light / standard / full
light
Use the provider's built-in settings for maximum output and timeouts, set by the person who configured the service.
standard
Bounds are set in the application or AI gateway per system and reviewed when usage changes.
full
The platform enforces bounds centrally per system and caller, with cost estimation before each call.

Enhanced 2 controls

Control 143

Set hard spend limits with anomaly alerting for denial of wallet.

EnhancedPreventiveall rolesper system

What it is

Per system and, where feasible, per caller, alerting as spend deviates rather than at billing

How to implement it · Minimum rules
  • Hard budget per system that stops or throttles usage at the cap
  • Budgets per caller where the provider or gateway allows it
  • Threshold alerts at 50%, 80% and 100% of budget
  • Spend-rate alert: hourly spend well above normal (e.g. three times the usual rate)
  • Alerts go to a named person with authority to suspend the offending caller; record each response
How to start

Set a monthly hard cap and an hourly spend-rate alert in the provider's billing settings for each production system.

Evidence

The budget limits, the spend-anomaly alert configuration and the alert-response records.

Mode: light / standard / full
light
Use the provider's billing caps and alerts, sent to the owner-manager's phone and email.
standard
Finance and the system owner share per-system budgets, with gateway-level per-caller limits and alerts to the operations team.
full
A cost-management function tracks spend in real time across providers, with anomaly detection feeding security monitoring.

Control 144

Restrict exposure of raw probabilities and other extraction-enabling detail in responses.

EnhancedPreventiveall rolesper system

What it is

Provide what the use case needs and no more of the model's internal signal

How to implement it · Minimum rules
  • Do not return token probabilities, raw scores or embeddings unless the use case needs them
  • Where a score is needed, round it or return a label or band
  • Do not expose system instructions, model configuration or internal reasoning traces to callers
  • Keep error messages generic: no stack traces, model names or configuration
  • Review the response format per use case and remove unused fields
How to start

Inspect the full response your service returns to callers and strip every field the interface or client does not use.

Evidence

The documented response format per use case and the configuration restricting detail.

Mode: light / standard / full
light
The person who set up the service turns off probability and detail options in the provider's settings and checks a sample response.
standard
Response formats are defined per integration and checked during release review.
full
The gateway enforces a response allow-list per use case, verified by automated tests.

Advanced 2 controls

Control 145

Detect and throttle model-extraction and cloning patterns on high-value models.

AdvancedDetectiveall rolesper system

What it is

Systematic query sweeps and abnormal harvesting volumes raise alerts and are rate-limited

How to implement it · Minimum rules
  • Apply to models worth stealing: ones you trained or adapted, or whose behaviour embodies valuable know-how
  • Baseline normal use per identity: volume, diversity of queries, timing
  • Detect systematic sweeps of the input space, near-template queries in volume, harvesting far beyond any task, coordinated accounts
  • Response: progressive throttling, alert to security monitoring, suspension pending review
  • Review alerts and wrongful throttles regularly and tune the thresholds
How to start

List which models are worth stealing, then compute each caller's daily query volume and diversity to spot outliers.

Evidence

The extraction-detection rules, their triggers and the throttling actions taken.

Mode: light / standard / full
light
A commissioned service or the provider's abuse detection watches usage, with the AI lead reviewing a monthly per-caller usage report.
standard
The gateway applies volume and diversity rules per identity and the security team reviews alerts.
full
Dedicated detection models score callers continuously, with automated throttling and investigation by the security operations centre.

Control 146

Design graceful degradation under load, with the shed order decided in advance.

AdvancedCorrectiveall rolesper system

What it is

The most important function is preserved; the design is tested, not assumed

How to implement it · Steps
  1. Rank the system's functions by importance and name the one that must stay alive
  2. Set the shed order, e.g. batch jobs and low-priority callers first, then shorter answers, then cached responses
  3. Set the triggers: load, latency, error rate or spend thresholds
  4. Build the mechanisms: priority queues, per-class quotas, circuit breakers, a fallback model
  5. Test by load test or simulated provider outage and record the result
How to start

Write the shed order for one critical system on one page with its owner, then test the first step under simulated load.

Evidence

The degradation design, its triggers and the dated test records.

Mode: light / standard / full
light
A one-page shed order agreed with the owner-manager, relying on the provider's priority and fallback settings and tested once.
standard
The engineering team builds the shed order into the application and gateway and tests it before major releases.
full
Degradation is part of the resilience architecture, tested in regular load and failure exercises with formal targets.

Evidence

The rate-limit and quota configuration bound to identity; the input, output, and cost bounds; the budget limits and spend-anomaly alerts; the response-detail restrictions; the extraction-detection rules and their triggers; the degradation design and its test records.