A connector update should preserve resource identity, content fidelity, deletion, and permissions. Test a realistic change trace against the old and new versions before treating successful authentication as a completed upgrade.
Connector regression testing covers lifecycle behaviour
A document connector translates source semantics into the assistant’s ingestion contract. Changes to pagination, metadata, extraction, or authentication can silently alter which resources become searchable.
For a proposed suite, create fictional documents and identities with known starting state. Apply edit, rename, move, permission change, delete, retry, and restart events. Record expected source and derived versions at each checkpoint.
Do not assume every source API or SDK change preserves ordering and checkpoint behaviour. Consult current official documentation for the exact resource and client version used in the implementation.
Replay a fictional metadata regression
The baseline connector preserves a restricted audience tag. The candidate imports text correctly but maps a missing field to public. Ordinary relevance tests can pass while the new version broadens access.
Another candidate changes resource-key construction after rename, creating duplicate index entries. The lifecycle suite should detect both the access failure and duplicate content, not merely compare total imported document count.
| Change event | Expected invariant |
|---|---|
| Content edit | Current representation replaces eligible old content |
| Rename or move | Documented identity semantics preserved |
| Permission removal | Supported revocation boundary holds |
| Source deletion | No late-job resurrection |
| Checkpoint recovery | No missing or uncontrolled repeated work |
Use controlled delays and failures so the test covers out-of-order responses and interrupted pagination. Preserve failed artifacts before rerunning.
Compare behaviour, not only schemas
A payload can retain its shape while a field’s meaning changes. Review source authority, timestamps, and permission semantics alongside parsing. Test final retrieval context and citations under fictional users.
Broad connector suites require maintenance, but they protect assumptions shared by many downstream questions. Keep a fast critical set and a fuller acceptance trace for material upgrades. A passing suite supports tested conditions rather than a claim that every source variation is covered.
This regression process is proposed. The voice integration case study describes related retrieval and backend work. Pair it with connector acceptance checklist.
Add a fixture where the candidate advances its checkpoint before all discovered changes are durably retained. A controlled restart should reveal the missing update and block acceptance.
Connector update FAQ
Is a successful test login enough?
No. Validate content, identity, permissions, lifecycle events, and recovery.
Should baseline failures be blamed on the candidate?
Label existing and newly introduced defects separately while preserving both as relevant risks.
What should block activation?
Use predefined material access, loss, duplication, fidelity, and recovery requirements for the supported collection.