Evaluation and release gates
Turn acceptance criteria into repeatable promotion decisions with recorded evidence and rollback conditions.
Before you start
Module prerequisites: SAI-100, SAI-120. Check the exercise-specific tools and environment before running commands.
Module outcomes: Map controls to technical evidence; Define audit events and evidence retention; Verify model, container, configuration, and decision provenance. This lesson contributes to that sequence.
A release gate is a decision system
An AI release gate turns requirements into a repeatable allow, reject, hold, or exception decision. It applies to models, prompts, retrieval pipelines, policies, runtimes, tools, infrastructure, and complete workload releases.
A gate is more than a test suite. It defines the candidate, acceptance criteria, evidence, decision authority, exception route, promotion destination, and rollback conditions.
Build the gate contract
For each gate, specify:
- Candidate identity: the immutable versions and configuration being evaluated.
- Approved purpose: workload, users, data classes, deployment boundary, and prohibited uses.
- Evaluation set: representative, boundary, adversarial, permission, failure, and regression cases.
- Measures: quality, safety, security, latency, capacity, reliability, cost, and policy outcomes that matter.
- Thresholds: pass, fail, warning, and manual-review criteria with justified tolerances.
- Decision authority: who can approve, reject, or accept an exception.
- Evidence: inputs, environment, results, logs, reviewer notes, decision, and integrity information.
- Aftercare: monitoring signals, rollback trigger, expiry, and required reassessment events.
Avoid metric theatre
A high aggregate score can hide an unacceptable failure for a specific user group, permission case, language, content type, or high-impact task. Keep disaggregated results and inspect failure severity as well as frequency.
Do not reuse a pass outside its evaluated context. A new prompt, knowledge corpus, quantization, runtime, policy, tool, hardware profile, or deployment boundary can change behavior and may require partial or complete reassessment.
Exceptions
An exception should identify the failed criterion, rationale, compensating controls, owner, affected scope, expiry date, monitoring, and closure evidence. Exceptions must not silently redefine the acceptance threshold.
Release evidence
The final record should allow an independent reviewer to reproduce the decision logic: what was tested, under which conditions, what failed, who decided, what was promoted, and how it can be rolled back.
Module worked example
A release has passing functional tests but no evidence for a required access-control objective.
- Apply the control
- Link the missing objective to an owner and hold the release pending evidence or a valid scoped exception.
- Expected result
- The gate records the unresolved finding and the specific condition needed to reconsider it.
- Troubleshooting
- Check evidence scope, revision and expiry. A high average coverage score cannot satisfy a missing critical control.
Practice and completion check
Choose one control from this lesson. Identify its actor, target, enforcement point and expected result. Describe an allowed case and a denied or failed case. For a runnable exercise, retain observed output and separate it from expected output. Record unresolved limitations before marking the lesson complete.
Your lesson progress
Completion is self-reported learning progress; it does not certify practical competence.
Status: Not marked
Loading your learning record…