The AI takes a long time to answer, but the transcript itself arrives late. Measuring only language-model generation would attribute the wait to the wrong stage.
For engineers profiling voice systems, separate input availability, turn completion, response generation and speech delivery. Each interval needs a named boundary and evidence from the component that performs the work.
What is transcription delay versus speech-generation delay?
Transcription delay concerns when usable text becomes available from incoming audio. Speech-generation delay concerns producing audio for the assistant's response. A pipeline may also spend time deciding that the caller's turn is complete and generating the response text.
Different architectures expose different events, particularly when one realtime service combines stages. Report the boundaries you can observe rather than invent separate timings for an opaque component.
Build a timing ledger for one turn
Use a fictional sentence with an agreed end point and preserve the configuration. The following proposed ledger connects measurements to the actual caller experience.
| Event | What the next interval may include |
|---|---|
| Caller finishes relevant speech | Input transport and recognition |
| Usable transcript available | Endpointing and turn commitment |
| Response generation begins | Model processing |
| Useful response text available | Speech generation |
| Audio becomes available | Publication and playback |
Some events overlap. Streaming can begin before the entire transcript or response is final, so do not simply add overlapping durations and call the sum end-to-end latency.
Inspect what the user actually waited for
A transcript partial may arrive quickly while the final interpretation remains unavailable. Likewise, the first audio byte may not contain a useful answer or may sit in a playback buffer before the caller hears it.
The LiveKit turn-taking guide helps identify the relationship between detection and response timing. The setup timeline guide covers the earlier telephone connection stage.
Keep tool work visible. A fast transcription followed by a slow calendar lookup is a different problem from a late transcript followed by immediate generation.
Compare repairs within the same workload
After changing one component, rerun representative turns with the same expected task outcome. Check recognition and saved details as well as timing, since a quicker partial interpretation can be incomplete.
My voice engineering case study provides project context, not a standard performance number. Publish measured distributions only with the workload, configuration, sample count and failure treatment needed to interpret them.
If a stage remains unobservable, record the combined interval and its limitation. A truthful broader measure is more useful than a precise-looking breakdown that the system never actually measured.
Can we add every reported duration together?
Only when the intervals do not overlap and their boundaries describe the same request path.
Is the first transcript partial a completed user turn?
Not necessarily. The application may still need more speech or a turn-completion decision.
What should we measure first?
Define observable input, generation and playback boundaries in the voice AI performance brief.