Baseline measurement
Baseline measurement for AI records how a defined workflow performs before an AI intervention. It establishes the reference point used to compare later changes in speed, quality, capacity, cost, or another approved outcome.
A baseline is more than an average copied from a dashboard. It identifies the work being measured, the observation period, the population, the metric formula, the source system, and the operating conditions that shaped the result.
Why does an AI initiative need a baseline?
Without a credible baseline, a team can report activity after launch but cannot show what changed. Higher output may reflect seasonal demand, staffing changes, easier work, a policy change, or another process improvement rather than the AI intervention.
The baseline also improves the AI business case. It replaces an assumed starting point with observed workflow performance and makes the expected improvement testable. It also gives AI ROI calculations a verified starting point instead of an assumed one.
HM Treasury's Magenta Book explains that impact evaluation compares observed outcomes with a counterfactual, meaning what would have happened without the intervention (Magenta Book). A pre-intervention baseline does not create that counterfactual by itself, but it is often a necessary input.
What should an AI baseline measure?
Define the unit of work first. Examples include an accepted support resolution, an approved contract review, a merged code change, or a completed invoice exception. Count work only when it reaches the agreed completion state.
Measure the outcome and its guardrails. A workflow baseline may include cycle time, accepted throughput, correction rate, escalation, defects, cost per completed unit, and labor input. The exact set depends on the decision. Avoid filling the baseline with every available metric.
Record adoption or tool usage separately. Those measures may explain exposure to the intervention later, but they do not describe the original workflow outcome.
How do you choose a baseline period?
Use a period long enough to capture normal variation and recent enough to represent the workflow being changed. Document seasonality, unusual demand, staffing shifts, policy changes, and incomplete data.
Segment the baseline when the work varies materially by complexity, channel, customer type, or team. A single blended average can make an easier post-launch case mix look like an improvement.
The baseline and post-launch measurement window should use compatible definitions. If the workflow, acceptance criteria, or source system changes, preserve the old definition and explain the break rather than silently joining unlike periods.
Baseline versus target and benchmark
A baseline describes current or prior performance. A target describes the desired future result. A benchmark compares performance with another team, organization, standard, or external reference.
These values can all support a decision, but they are not interchangeable. A vendor benchmark may show what appears possible while saying little about the organization's starting point. A target may express ambition without proving that the improvement occurred.
NIST's AI Risk Management Framework calls for expected benefits and costs to be compared with appropriate benchmarks and for experimental design and data collection choices to be documented (NIST AI RMF Core).
How does a baseline support benefits realization?
Benefits realization tracks whether the operational changes required by the investment actually occur. Each benefit should retain its original baseline, target, owner, measurement method, and review date.
When results arrive, compare actual outcomes with the baseline and with a fair counterfactual where possible. Then separate observed change from realized AI value, which also requires evidence that the organization captured the improvement in operations.
Common baseline-measurement mistakes
Common failures include measuring only enthusiastic users, using a period distorted by an incident, changing metric definitions after launch, ignoring work complexity, and reconstructing the baseline after results are known.
The strongest baseline is agreed before deployment and reproducible from source records. An AI measurement framework records the source, owner, formula, period, and limitation for each metric. If the available data is weak, record the limitation and improve the measurement design before making a large financial claim. An AI measurement readiness assessment tests whether the workflow can produce a credible baseline.