Data and control boundary
Map inference, embeddings, retrieval, telemetry, backups, support access, and updates. On-premises hosting alone does not establish sovereignty; document who can access each component and how that access is enforced.
Interactive guide / Architecture & procurement
Model Selection Guide
Choose an AI model you can operate, evaluate, and afford. Build a shortlist around your data boundary, task, runtime, and cost per accepted outcome, then validate it on your own workload.
Reviewed · Vendor-neutral guidance
- Decision
- A defensible model shortlist and a clear pilot decision.
- For
- Platform leads, architects, engineering teams, and technical buyers.
- Available here
- Interactive architecture guidance and an editable selection scorecard.
Build your candidate path
Your deployment choice is a constraint. Missing hardware prompts a feasibility review; it never silently changes an on-premises requirement to cloud hosting. Inputs stay in this page and are not submitted.
Candidate path; validate feasibility
On-premises: Document RAG
Compare a compact instruction model with a stronger candidate using the same authorized retrieval and citation tests.
Benchmark a compatible vLLM or equivalent serving release on the specified GPUs; confirm exact model, quantization, and feature support.
Required checks
- Evaluate the actual English document styles, accents if applicable, and user tasks.
- Set explicit input and output token budgets, including retrieved context and tool responses.
- Review control of keys, administrators, updates, telemetry, and artifacts across the entire stack.
- Retain versioned decisions and redacted evidence with access and retention controls; avoid indiscriminate raw prompt logging.
Constraints and assumptions
- This is a candidate architecture, not a capacity, compliance, or sovereignty certification.
01 / Decision framework
Apply the constraints before ranking the models
A high benchmark score cannot compensate for an unacceptable data path or an unsupported production runtime. Treat these as pass/fail gates before comparing quality.
Rights and reproducibility
Record the exact model repository and revision, tokenizer, chat template, weight format, license, and acceptable-use terms. Check commercial use and redistribution for your intended deployment. Open weights do not automatically mean unrestricted use.
Runtime compatibility
Check the selected architecture, quantization, accelerator, tool-call parser, and multimodal support against the pinned serving release. A compatible API shape does not guarantee identical behavior across providers or runtimes.
Operating ownership
Name the owner for patching, access review, capacity, incidents, model replacement, and rollback. Include support and evaluation effort in the comparison with a managed service.
02 / Decision framework
Shortlist by the job the system must do
Compare a simple baseline, a cost-conscious candidate, and a quality-focused candidate within the approved boundary. Keep the dataset and acceptance rules identical.
| Workload | Starting shortlist | Evidence to collect |
|---|---|---|
| Document RAG | Retriever + compact instruction model; larger model only if failures justify it. | Authorized retrieval, claim support, citation coverage, abstention, and cost per accepted answer. |
| Extraction / classification | Rules or a trained classifier alongside a small language model. | Per-field accuracy, schema validity, rare-class recall, and human correction effort. |
| Code assistance | Code-capable model plus permission-aware repository retrieval. | Held-out executable tests, insecure changes, license constraints, and sandbox behavior. |
| Agents and tools | Tool-capable model behind explicit tool permissions and approval rules. | Completed authorized tasks, denied actions, loop limits, recovery, and total task cost. |
| Vision / sensor | Task-specific vision or signal model; multimodal model where its flexibility is needed. | Device-level latency, false alarms, missed events, energy, and field robustness. |
Current example: Qwen3.5-9B combines vision and language with hybrid attention. It is a candidate to test, not a universal recommendation. Its memory behavior differs from a conventional full-attention 9B model; use its model card and the serving runtime's measurements.
03 / Decision framework
Turn a shortlist into an approval decision
Agree the decision rules before running the pilot so that a successful demo does not become the only acceptance test.
Freeze a representative holdout set, including languages, long inputs, missing evidence, and unauthorized requests. Keep development examples separate.
Set quality and security gates first. Measure p95 time to first token, end-to-end latency, failure rate, and completed work under realistic simultaneous demand.
Test the exact quantized artifact and decoding settings you will deploy. Include reasoning output, retries, tool calls, and evaluator costs in the budget.
Record a primary candidate, an approved fallback, rejected alternatives, and the conditions that trigger reassessment. Re-run the suite after model, prompt, or runtime changes.
Advertised context length is a technical limit, not evidence of reliable recall across that entire length. Test the locations and document lengths that matter to your users.
Working materials
Take the framework into your project
Download the editable materials, adapt them to your environment, and keep the assumptions alongside the results.
Work with SovAIHub
Get a model decision your team can defend
Scope a model selection assessment around one priority workload and its real deployment constraints.
- A boundary and license review with explicit assumptions and unresolved decisions.
- A versioned candidate matrix and workload-specific evaluation plan.
- A measured pilot comparison where environment access and test data are included in scope.
- A decision record covering runtime, hardware, costs, fallback, and next implementation steps.
Bring to the scoping conversation
Share the use case, permitted data locations, sample task shapes, languages, expected demand, available hardware, and the person who accepts the result.
Scope, acceptance criteria, delivery schedule, and fees are agreed before implementation.
Request a model selection reviewContinue your implementation
Connect the decision to the next engineering or operating step.
Basic coding agent lab →Reviewed model candidates →Size the hardware →Evaluate RAG quality →Compare operating costs →
Technical references
Official documentation checked for this revision. Confirm the exact model, runtime, and license versions when implementing.