A missing-field test should prove that the assistant asks for the right information and that no incomplete write occurs. Checking only whether a schema rejects the payload leaves the conversation recovery untested.
Missing is different from empty or invalid
Required-field testing removes or corrupts one necessary input while keeping the remaining request realistic. Distinguish absent values, explicit nulls, blank strings, malformed values, and unavailable lookup results. They may need different explanations.
In a proposed fixture set, define the minimum valid payload first. Derive one case per required field, then add combinations that commonly occur together. Record the expected validation category, permitted next question, and forbidden side effects.
The JSON Schema object reference explains that declaring properties does not by itself make them required. Confirm that the actual validator and schema version enforce the intended contract.
Test a fictional address gap
A caller gives a name, callback number, and repair description but no service address. The tool should return a missing-address result without creating a dispatchable request. The agent should ask for the service location, rather than ask the caller to repeat every detail.
Now change the fixture to an address containing a street but no unit number. Whether the unit is required depends on the business rule and location. The assistant should not fabricate “Unit 1” to satisfy a schema.
| Fixture | Expected recovery |
|---|---|
| Address property absent | Ask for the service address |
| Address is blank | Treat it as unresolved information |
| Callback lookup unavailable | Ask through the approved fallback |
| Two required fields missing | Gather both without losing existing facts |
| Optional note absent | Continue if other prerequisites hold |
After clarification, verify that the corrected request carries the new value and preserves previously confirmed fields. A recovery path that rebuilds the whole payload can accidentally discard a timezone or customer reference.
Observe the write boundary
Use a fake execution service that records every attempted mutation. Negative cases should assert zero writes and zero downstream notifications. The assistant may still perform allowed read-only lookups, so distinguish mutation counts from total tool calls.
Testing every possible field combination becomes expensive as schemas grow. Cover single-field failures systematically, then choose interaction cases from actual domain dependencies. Keep a small set of multi-field scenarios that exercise conversation memory.
This test design is proposed. The HVAC request workflow documents collecting service details and confirmation; these fixtures do not claim to be its existing tests. Compare argument validation for the broader boundary.
Missing-field FAQ
Is a null value equivalent to an absent field?
Only if the contract defines it that way. Test both representations explicitly.
Should the assistant fill in defaults?
Use only documented defaults that preserve the caller’s intent. Do not invent consequential information.
What makes the recovery pass?
The agent asks a relevant question, retains confirmed context, and writes only after the required information is valid and authorised.