A pilot should end with a specific decision: expand a defined scope, revise and retest, gather more evidence, or stop. Agree on the decision criteria before attractive demos or sunk costs influence the judgment.
Expansion requires operational readiness
A pilot decision combines correct outcomes, critical defects, staffing, cost, and evidence quality. A high answer rate alone does not establish that the service is safe to expand or useful enough to justify continued effort.
For a proposed decision sheet, list the original question, supported scope, required invariants, observed results, unresolved work, and next-stage capacity. Separate failures that demand correction from uncertainties that require more data.
Keep accountable business and engineering owners involved. Operations understands staffing and follow-up consequences; engineering can establish whether writes, permissions, and recovery behave correctly. Neither view alone covers the whole workflow.
Compare fictional pilot choices
A pilot saves accurate requests but produces occasional unsupported appointment promises. The useful intake result does not cancel the correctness defect. The team might restrict the commitment language and retest before expanding, under its agreed release policy.
Another pilot has no observed critical defects but only a handful of eligible calls. The appropriate decision may be a longer bounded observation window, not a broad claim of proven reliability. These are illustrative decision patterns, not reported deployment outcomes.
| Decision | Evidence and condition |
|---|---|
| Expand | Gates met and added scope clearly bounded |
| Revise and retest | Specific defect has an owner and verification plan |
| Extend observation | Material uncertainty can be reduced with defined data |
| Stop | Required capability or value is unsupported under the scope |
| Pause affected workflow | Containment protects callers while review continues |
Record what would change the decision. “Continue improving” without a question, owner, or endpoint can turn a pilot into indefinite unsupported operation.
Include the work created for staff
Review correction time, callback backlog, incident response, and maintenance. A system may handle more calls while increasing total operational burden. Conversely, a modest financial result may still support a limited service goal if the owner explicitly values it.
Expansion changes traffic, case mix, and capacity. Preserve monitoring and regression checks, validate new routes and staff roles, and define containment before increasing exposure. A successful narrow pilot is evidence for that scope, not every future workflow.
This decision framework is proposed. The HVAC case study demonstrates an independent workflow without claiming pilot economics. Read owner-readable pilot reports.
Pilot decision FAQ
Should a good average score override a critical defect?
Use the predeclared gates. Do not average away material unauthorised or misleading outcomes.
When is more testing justified?
When it addresses a specific unresolved question with a defined observation or verification plan.
What makes expansion reviewable?
State the added scope, evidence, owners, capacity, monitoring, unresolved limits, and conditions for reducing traffic again.