AI threat-model workshop
Produce an asset inventory, attack-surface map, prioritized scenarios, control plan, evidence plan, and residual-risk register.
Before you start
Module prerequisites: SAI-100. Check the exercise-specific tools and environment before running commands.
Module outcomes: Draw system and data-flow boundaries; Identify AI-specific threats and abuse paths; Translate threat scenarios into control requirements. This lesson contributes to that sequence.
Workshop objective
Create a traceable threat model for the synthetic private knowledge assistant introduced in SAI-100. Use synthetic documents and fictional identities. Do not test production systems.
Download SAI-110 threat-register templateA CSV register for traceable scenarios, controls, evidence, residual risk, owners, and state.Step 1: Confirm scope
Review the system definition, prohibited uses, actors, lifecycle, context diagram, trust zones, and numbered flows. Record exclusions and the person authorized to accept them.
Exit check: reviewers agree on what system and version is being modelled.
Step 2: Inventory assets and surfaces
Use AI assets and attack surfaces. Include documents, permissions, index, prompts, model, packages, identities, gateways, operators, evidence, backups, and recovery capability.
Exit check: each high-value asset has an owner, required property, location, boundary, interface, and recovery need.
Step 3: Model actors and abuse cases
Include an external actor, authorized user, privileged operator, compromised service identity, supplier or artifact source, and unintended misuse case. Remove actors that cannot reach any surface.
Exit check: each abuse case identifies starting access, action, boundary, asset, and consequence.
Step 4: Develop scenarios
Create at least eight end-to-end scenarios across:
- Artifact or model supply chain.
- Document ingestion and retrieval permissions.
- Prompt, context, and output handling.
- Identity, gateway, or routing bypass.
- Resource exhaustion or service disruption.
- Privileged operation and evidence integrity.
- Backup, restore, update, or retirement.
Exit check: every scenario references diagram flow IDs and asset IDs.
Step 5: Prioritize and treat
Apply the same impact, feasibility, reach, detectability, recoverability, control-strength, and uncertainty definitions to every scenario. Select treatment for the highest-priority scenarios.
Exit check: planned controls have not been counted as implemented controls.
Step 6: Define tests and evidence
For the five highest-priority scenarios, define:
- A negative test for the prohibited or unsafe path.
- A positive test for approved behavior.
- Expected policy decision and system response.
- Required version identifiers.
- Evidence produced, protected, and reviewed.
- Containment and recovery action.
Workshop package
Deliver:
- Scope, assumptions, and exclusions.
- Asset-to-attack-surface inventory.
- Actor and abuse-case register.
- Prioritized threat-scenario register.
- Control and verification plan.
- Residual-risk register.
- Threat-driven readiness backlog.
The workshop is an educational design review. It is not a penetration test or production security authorization.
Module worked example
A retrieved document contains instructions to disclose a private file.
- Apply the control
- Treat the text as untrusted input and identify the separate file-access and egress controls that must deny the action.
- Expected result
- The threat register connects the abuse case to enforcement, a negative test and residual risk.
- Troubleshooting
- If the mitigation is only a prompt instruction, identify what prevents the side effect outside the model.
Practice and completion check
Choose one control from this lesson. Identify its actor, target, enforcement point and expected result. Describe an allowed case and a denied or failed case. For a runnable exercise, retain observed output and separate it from expected output. Record unresolved limitations before marking the lesson complete.
Your lesson progress
Completion is self-reported learning progress; it does not certify practical competence.
Status: Not marked
Loading your learning record…