Observed value
Observed AI value is a measurable improvement seen after an AI-enabled change, before the organization has fully established attribution or benefit capture. It records what changed in the workflow without claiming that AI caused all of the change or that the organization realized the full economic benefit.
Observed value is an evidence stage. It is stronger than a forecast and weaker than attributed or realized value.
What counts as observed AI value?
Observed value can include a change in accepted throughput, cycle time, correction rate, rework, quality, service level, cost per completed unit, or another approved outcome.
The measure should use a defined population, unit of work, baseline, measurement window, and source. For example, a team may observe that median cycle time fell for eligible cases after deployment. That statement does not yet establish why the change occurred.
Activity alone is not observed value. More prompts, active users, model calls, or generated drafts show use. They become relevant to value only when connected to a workflow outcome.
HM Treasury's Magenta Book distinguishes monitoring of inputs, activities, outputs, and outcomes and explains how those records contribute to evaluation (Magenta Book).
Observed value versus expected value
Expected value is the benefit forecast in an investment case. It depends on assumptions about volume, adoption, performance, cost, timing, and organizational change.
Observed value uses actual post-intervention records. Compare it with the forecast, but keep the two series labeled. A pilot may show an improvement smaller than expected, delayed, limited to one segment, or accompanied by a quality loss.
AI opportunity sizing should use observed evidence from similar work when available. Historical observations can improve future estimates, but differences in workflow and context must remain visible.
Observed value versus attributed impact
Observed value answers what changed. Impact attribution asks how much of that change occurred because of the AI intervention.
Other factors may include staffing, demand, case mix, policy, seasonality, training, or process redesign. A before-and-after chart can support monitoring but cannot remove those explanations by itself.
HM Treasury's Quality in Policy Impact Evaluation guidance states that monitoring can identify a change, while causal attribution requires a counterfactual and an appropriate comparison design (QPIE guidance).
Observed value versus realized value
Realized AI value requires evidence that the organization captured the supported benefit after relevant costs and disbenefits. Observed capacity is not automatically realized financial value.
If a team completes work faster, the organization may use the capacity for more accepted work, better service, avoided hiring, or another approved outcome. Until that operating change is documented, report the measured improvement in its original unit.
How should observed value be recorded?
Store the metric definition, baseline, observation period, eligible population, actual result, quality guardrails, cost boundary, and important contextual changes. State whether the comparison is descriptive or designed for attribution.
Link the result to the AI initiative inventory and the original AI pilot success criteria. Preserve unfavorable and inconclusive observations rather than reporting only metrics that improved.
NIST's AI Risk Management Framework recommends documenting measurable performance improvements or declines using field data and relevant stakeholder input (NIST AI RMF Core).
Common observed-value mistakes
Common failures include reporting activity as value, selecting the best metric after launch, ignoring quality or risk, blending unlike populations, and presenting temporal sequence as causation.
Another mistake is converting every observed time improvement into money. Keep operational and financial measures separate until finance validates the capture method and the organization demonstrates how the capacity was used.