Configuration can change agent behavior as much as code. A wrong agent identity, business mapping, or escalation destination can break production while the executable remains unchanged.
LiveKit's testing overview separates development and evaluation tools, and its unit-testing guide describes turn-level assertions and tool mocking. Use each layer for what it can verify, then add controlled integration checks for destinations and telephony.
Version the complete configuration under review
Record policy, client facts, knowledge sources, action schemas, environment mappings, model selection, and routing references. Keep secrets outside ordinary configuration history and refer to approved secret identities rather than values.
Validate required fields and allowed destinations before execution. A structurally valid configuration can still point at the wrong client, so include semantic fixtures and negative tests. Do not assume a schema check establishes business correctness.
Work through a fictional calendar mapping change
A release changes one client's calendar reference. Turn-level tests verify clarification and honest failure language using controlled outcomes. An integration fixture verifies the actual destination and resulting record under the intended account.
A phone-path rehearsal checks that the number reaches the intended agent and business configuration. The acceptance record identifies which evidence came from mocks and which came from real controlled dependencies.
| Configuration test | What it proves |
|---|---|
| Required-field validation | Expected shape and values |
| Tool-argument assertion | Intended action request |
| Mocked rejection | Conversation fallback behavior |
| Controlled remote write | Actual mapping and permission |
| Phone-path fixture | Routing and agent selection |
| Foreign-client identifier | Enforced authorization boundary |
Review limits before production
Text-based checks cannot establish microphone capture, audio timing, or SIP-provider behavior. Full phone-path tests are slower and can incur usage, so target consequential changes and isolate side effects.
A broad suite can become expensive and difficult to interpret. Prioritize changed boundaries, common paths, costly wrong actions, and recovery. Keep exact pass evidence and unresolved differences instead of one undifferentiated green status.
Rerun affected tests after SDK, provider, permission, or business-rule changes. Review fictional telephony fixtures for controlled end-to-end evidence.
Questions about preproduction testing
Does a mocked tool prove the live API works?
No. It verifies behavior under the mocked result. Actual permissions, destinations, and remote semantics need controlled integration evidence.
Should every release run every expensive audio test?
Use a risk-based plan and the agreed acceptance scope, with broader checks when consequential boundaries change.
What should the reviewer approve?
The exact configuration version, observed outcomes, environment differences, known limits, rollout and rollback plan, and operational ownership for unresolved requests.