A false booking confirmation occurs when the assistant tells the caller an appointment exists without authoritative evidence. Measure that claim directly; an eventual successful write does not excuse a premature promise.
Define the claim before measuring it
False booking confirmation testing compares caller-facing assertions with the booking system’s state at the time of the assertion. Include direct phrases such as “you are booked” and equivalent implications such as “the technician will arrive then” when they communicate a committed appointment.
For a proposed evaluation, annotate each consequential claim with its timestamp and referenced appointment details. Compare it with the authoritative result available by that moment. Distinguish confirmed appointments, pending requests, rejected bookings, and uncertain writes.
Use two denominators when helpful: the share of booking conversations containing an unsupported confirmation, and the share of confirmation claims that are unsupported. State which one is being reported so a conversation with repeated claims does not distort interpretation.
Calculate a fictional test result
Suppose 50 scripted booking conversations produce 40 completion claims. Three claims occur before any confirmed booking result, and one claim follows a rejected write. The unsupported-claim rate is four out of 40 claims, or 10 percent.
If those four claims occur in three conversations, the affected-conversation rate is three out of 50, or 6 percent. These numbers are illustrative. They describe different questions and should not be merged into one unlabeled accuracy score.
| Backend state at claim time | Evaluation |
|---|---|
| Confirmed appointment with matching details | Supported confirmation |
| Request accepted into a queue | Appointment confirmation unsupported |
| Write rejected | Confirmation false |
| Execution status unknown | Definitive confirmation unsupported |
| Appointment exists at a different time | Claimed details false |
Include cases where the assistant later corrects itself. Record the correction, but retain the original unsupported claim as a defect because the caller may have acted on it or disconnected first.
Pair automated checks with human review
Structured backend assertions can establish the appointment state. Detecting implied promises in varied speech may require reviewed labels. Use a sample of both flagged and unflagged conversations to estimate what the detector misses.
A strict detector can flag harmless statements about intent, such as “I will check that time.” Keep the annotation rule focused on claims of completed business commitment. Review disagreements before using the metric as a release gate.
This evaluation is proposed. The HVAC case study makes the request-versus-appointment boundary explicit. Read conversation versus task scores for reporting both dimensions.
False confirmation FAQ
Does eventual booking success make an early claim correct?
No. The claim needs supporting evidence when it is made.
Is a saved service request a booking?
Only if the business explicitly defines it as a confirmed appointment. Many intake systems do not.
Should corrected false confirmations still count?
Yes. Track recovery separately so the initial failure remains visible.