Controlled change and versioning
Version complete AI behavior, assess change impact, approve promotion, support rollback, and retire superseded assets.
Before you start
Module prerequisites: SAI-100. Check the exercise-specific tools and environment before running commands.
Module outcomes: Design a private ingestion and retrieval flow; Implement permission-aware retrieval; Evaluate grounding, citations, and no-answer behavior. This lesson contributes to that sequence.
Version the behavior, not only the model
An AI system's behavior depends on a configuration set: model and tokenizer, prompt and templates, retrieval corpus and index, policy, tools and schemas, runtime, dependencies, hardware profile, feature flags, and application code. A release identifier should resolve to that complete set.
Classify change
Define change classes based on possible impact, not file size. A one-line policy or prompt update can be material. For each class, specify required review, evaluation, approval, deployment, monitoring, rollback, and evidence.
Common reassessment triggers include:
- Model, quantization, tokenizer, runtime, or hardware changes.
- Prompt, tool, retrieval, corpus, permission, or policy changes.
- New users, purpose, data class, geography, provider, or deployment boundary.
- Security findings, incidents, drift, quality regression, license changes, or expired exceptions.
Controlled promotion
Identify the immutable candidate and its parents. Evaluate in a representative environment, record results and exceptions, obtain authorized approval, promote without rebuilding, verify deployment identity, monitor the defined window, and preserve rollback.
Separate content movement from authorization. The ability to copy an artifact must not imply permission to promote or execute it.
Rollback and forward recovery
Rollback must consider data schemas, indexes, caches, state, secrets, and evidence compatibility as well as binaries. Some changes are not safely reversible; document forward-recovery steps and the point of no return.
Retirement
Remove superseded artifacts from active selection, revoke credentials and approvals where appropriate, preserve required evidence, update dependencies, and verify that hidden routes cannot still invoke the retired version.
Record architecture consequences and reversal triggers in an architecture decision record. Tool-specific deployment syntax belongs in separately tested adapters; the change contract remains stable across platforms.
Module worked example
Two users ask the same finance question, but only one may read the source document.
- Apply the control
- Filter active documents by the caller's permissions before retrieval context is assembled.
- Expected result
- The permitted actor gets a supported citation; the denied actor receives no finance context.
- Troubleshooting
- Inspect retrieved document IDs and cache identity. Removing a citation after generation cannot undo disclosure.
Practice and completion check
Choose one control from this lesson. Identify its actor, target, enforcement point and expected result. Describe an allowed case and a denied or failed case. For a runnable exercise, retain observed output and separate it from expected output. Record unresolved limitations before marking the lesson complete.
Your lesson progress
Completion is self-reported learning progress; it does not certify practical competence.
Status: Not marked
Loading your learning record…