A before-and-after comparison is meaningful only when the outcome definitions remain stable. Changing what counts as eligible or complete can create an apparent improvement without changing the caller’s experience.
Comparability starts before data collection
A call-outcome comparison evaluates two periods using the same eligibility rules, business stages, observation window, and cost scope. Record those definitions before the pilot and apply them retrospectively where feasible.
For a proposed analysis, match weekday mix, staffed periods, locations, and supported workflows. Annotate marketing changes, seasonality, outages, and policy changes. These factors can affect volume and conversion independently of the receptionist.
Do not interpret a simple comparison as proof of causation. It can describe an observed change while leaving uncertainty about what would have happened without the intervention.
Recalculate a fictional misleading improvement
Before the pilot, 40 of 80 eligible calls produce correctly saved requests, or 50 percent. During the pilot, a report shows 48 successes out of 60 calls, or 80 percent, but it excluded 20 difficult valid calls as “unclear.”
Restoring the same eligibility definition gives 48 out of 80, or 60 percent. That is still an observed increase of ten percentage points, but materially different from the original claim. The numbers illustrate denominator discipline, not measured results.
| Comparison element | Consistency check |
|---|---|
| Eligible call | Same exclusion and unknown rules |
| Completed outcome | Same authoritative business stage |
| Repeat contact | Same deduplication unit |
| Conversion window | Equal time for outcomes to mature |
| Cost | Same included categories and period basis |
Keep raw counts and a reconciliation table showing exclusions. A reviewer should be able to recreate the denominator rather than trust an unexplained dashboard percentage.
Account for immature and missing outcomes
Recent enquiries may not yet have resolved. Compare cohorts with equivalent follow-up windows or mark outcomes as pending. Missing baseline data should remain a limitation, not be filled with convenient assumptions.
Stricter comparisons may reduce sample size or delay conclusions. That tradeoff is preferable to presenting incompatible cohorts as a precise improvement. If practical, a controlled rollout or matched comparison can add stronger evidence, with its own design limits.
This analysis method is proposed. The voice integration case study provides related context without claiming causal business gains. See missed calls versus new customers.
Before-and-after FAQ
Is a higher percentage always evidence of improvement?
Check counts, definitions, exclusions, case mix, and observation windows first.
Should unknown calls be removed?
Use the same documented rule in both periods and report the unknown category visibly.
Can a before-and-after report prove revenue attribution?
Not by itself. Attribution requires additional evidence and explicit assumptions about other influences.