Cost & value / Private AI operating economics
Connect AI spend to useful work. Compare managed services and self-hosted inference on a consistent cost basis, expose idle capacity, and improve cost per accepted outcome while protecting quality, service levels, and data boundaries.
Reviewed · Vendor-neutral guidance
01 / Decision framework
Choose a currency, reporting period, workload boundary, and accounting basis before comparing options. Keep actuals, estimates, and scenarios visibly separate.
| Cost layer | Managed or consumption-based | Owned or reserved infrastructure |
|---|---|---|
| Inference | Input/output tokens, cache reads/writes, batch rates, provisioned capacity, and billable reasoning where applicable. | Lease or depreciation allocation, reserved capacity, accelerators, host compute, and idle time. |
| Application pipeline | Embeddings, reranking, retrieval/storage, tool APIs, retries, and network transfer. | Equivalent retrieval, storage, networking, adapters, and shared platform services. |
| Operations and quality | Integration, evaluation/judging, human review, monitoring, support, and incident effort. | The same functions plus drivers, patching, scheduling, spares, and capacity management. |
| Boundary and resilience | Required regions, contractual controls, retention settings, dedicated tenancy, and service redundancy. | Facility energy, cooling allocation, backup power, offline mirrors, security controls, and failover capacity. |
For an amortized operating view, do not add the full hardware purchase and depreciation for the same asset. For a cash-flow view, use purchase timing instead. Apply PUE to IT energy only when facility overhead is not already included in the electricity allocation.
02 / Decision framework
Period cost per accepted outcome = attributable period cost / accepted outcomes in the same period. Keep the acceptance definition stable and report the sample and rejection counts.
Include retrieval, generation, evaluation, retries, and review within the agreed boundary. Define acceptance using task correctness, evidence, permissions, and response-time targets. Zero accepted answers makes unit cost undefined; report the spend and the failure.
Count whole workflows, including unsuccessful loops, tool fees, supervision, and recovery. Attribute all attempts to the workload; count only valid completed outcomes in the denominator. A cheap tool call is not necessarily a cheap business task.
Measure idle allocation, queue delays, achieved throughput, and useful work meeting the agreed service targets. GPU utilization alone can rise while quality, user experience, or cost per outcome deteriorates.
Separate measured spend from assumptions about time saved or revenue. Validate labor savings with observed adoption and handling effort. Report forecast ranges and payback sensitivity rather than treating every generated token as value.
03 / Decision framework
Use one allocation rule per cost pool and preserve the source records. A reproducible estimate is more useful than a precise-looking dashboard with unexplained totals.
Inventory workloads, providers, models, environments, owners, and cost centers. Join billing exports and resource costs to request or task counts using privacy-preserving identifiers.
Allocate direct costs first. Distribute shared costs using a documented driver such as measured GPU-seconds or reserved capacity. Keep idle and unallocated costs explicit and reconcile to the period total.
Run low, expected, and high-demand scenarios with the same quality, latency, availability, and boundary constraints. Include migration and operating effort and test how utilization changes the result.
Prioritize one optimization at a time: retrieval and prompt efficiency, smaller-model routing, caching, batching, scheduling, or placement. Re-evaluate quality and security, then compare measured accepted-outcome cost.
FOCUS can help normalize compatible billing exports; custom workload, quality, and owned-hardware allocations still need explicit definitions. This worksheet is a planning template, not a claim of FOCUS conformance.
Working materials
Download the editable materials, adapt them to your environment, and keep the assumptions alongside the results.
Work with SovAIHub
Scope a FinOps assessment around a defined set of workloads and a decision such as rightsizing, sourcing, showback, or production expansion.
Bring billing exports, hardware or lease costs, energy assumptions, traffic and token volumes, accepted-outcome counts, service targets, and a finance owner for allocation decisions.
Scope, acceptance criteria, delivery schedule, and fees are agreed before implementation.
Scope an AI FinOps assessmentConnect the decision to the next engineering or operating step.
Official documentation checked for this revision. Confirm the exact model, runtime, and license versions when implementing.