A low-latency session can still fail to book the customer's visit. Measure the conversation pipeline and the business workflow separately, then connect them through stable identifiers.
LiveKit documents session traces and metrics and data hooks for session-level evidence. Use that evidence to locate delays and execution stages. Your application must still define what counts as a verified customer outcome.
Add an action-level outcome model
Record call or session reference, client, request identity, intended operation, attempt time, remote identifier, result category, and follow-up owner. Distinguish explicit failure from unknown outcome and from a confirmed result.
Do not infer a successful booking merely because a tool returned text or the agent spoke confirmation. Inspect the destination evidence required by the business. A successful API response with the wrong property is a semantic failure, not a latency problem.
Work through a fictional fast but failed call
A caller receives quick responses and agrees to a visit. The scheduling action times out after submission. The agent correctly says confirmation is pending, and the request enters reconciliation.
The report should show acceptable conversational responsiveness alongside an unresolved booking action. If the remote appointment is later found, the action becomes confirmed with evidence. It should not retroactively imply the caller heard a verified confirmation during the call.
| Observation | Correct interpretation |
|---|---|
| Fast first response | One conversational performance measure |
| Tool timeout | Action outcome may remain unknown |
| Remote record verified | Business result confirmed |
| Transfer attempted | Staff acceptance still unproven |
| Session closed normally | No guarantee all requests resolved |
Report exceptions operators can act on
Prioritize uncertain writes, unaccepted handoffs, duplicate candidates, missing destination links, and aging callbacks. Include current owner and next action. Pipeline timing charts help diagnosis, while these queues help finish customer work.
More event detail improves investigation but adds storage and privacy responsibilities. Keep ordinary logs minimal and use controlled source records for deeper review. Define metric denominators and reporting cutoffs before comparing releases.
Use fictional fixtures to validate event joins and partial outcomes. Avoid publishing accuracy or latency improvements without an actual measurement method and results. Review outcome dashboard design for the operator-facing view.
Questions about workflow metrics
Can session completion be the success metric?
It is useful lifecycle evidence, but it should not replace verified booking, accepted handoff, or resolved-request outcomes.
Should every timeout count as a failed booking?
Keep uncertain outcomes separate until the destination is reconciled. A timeout may hide a successful remote write.
What should a release comparison include?
The same scenario definitions, workload conditions, outcome evidence, unresolved work, and measurement window. Separate measured results from proposed targets.